4.6 Breaking the abstraction

In What makes Lisp different, we mentioned that writing an FFI sometimes requires depending on information not provided as part of the interface. The easy option CURLOPT_WRITEDATA, which we will not provide as part of the Lisp interface, illustrates this issue.

Strictly speaking, the curl-option enumeration is not necessary; we could have used :int 99 instead of curl-option :nosignal in our call to curl_easy_setopt above. We defined it anyway, in part to hide the fact that we are breaking the abstraction that the C enum provides. If the cURL developers decide to change those numbers later, we must change the Lisp enumeration, because enumeration values are not provided in the compiled C library, libcurl.so.3.

CFFI works because the most useful things in C libraries — non-static functions and non-static variables — are included accessibly in libcurl.so.3. A C compiler that violated this would be considered a worthless compiler.

The other thing define-curl-options does is give the “type” of the third argument passed to curl_easy_setopt. Using this information, we can tell that the :nosignal option should accept a long integer argument. We can implicitly assume t == 1 and nil == 0, as it is in C, which takes care of the fact that CURLOPT_NOSIGNAL is really asking for a boolean.

The “type” of CURLOPT_WRITEDATA is objectpoint. However, it is really looking for a FILE*. CURLOPT_ERRORBUFFER is looking for a char*, so there is no obvious CFFI type but :pointer.

The first thing to note is that nowhere in the C interface includes this information; it can only be found in the manual. We could disjoin these clearly different types ourselves, by splitting objectpoint into filepoint and charpoint, but we are still breaking the abstraction, because we have to augment the entire enumeration form with this additional information.¹

The second is that the CURLOPT_WRITEDATA argument is completely incompatible with the desired Lisp data, a stream.² It is probably acceptable if we are controlling every file we might want to use as this argument, in which case we can just call the foreign function fopen. Regardless, though, we can't write to arbitrary streams, which is exactly what we want to do for this application.

Finally, note that the curl_easy_setopt interface itself is a hack, intended to work around some of the drawbacks of C. The definition of Curl_setopt, while long, is far less cluttered than the equivalent disjoint-function set would be; in addition, setting a new option in an old libcurl can generate a run-time error rather than breaking the compile. Lisp can just as concisely generate functions as compare values, and the “undefined function” error is just as useful as any explicit error we could define here might be.