Lisp and C cannot in general share memory so the FLI needs to make a copied of strings, either temporarily when passing them to C or as new Lisp objects when returning them.
Use of the :reference-pass type in this example converts the Lisp string to a foreign string on calling, but does not convert the string back again on return.
Here is the C code for the example. It uses the argument string but returns an integer.
#include <string.h>
#include <ctype.h>
__declspec(dllexport) int __cdecl count_upper(const char *string)
{
int count;
int len;
int ii;
count = 0;
len = strlen(string);
for (ii = 0; ii < len ; ii++)
if (isupper(string[ii]))
count++;
return count;
}
#include <string.h>
#include <ctype.h>
int count_upper(const char *string)
{
int count;
int len;
int ii;
count = 0;
len = strlen(string);
for (ii = 0; ii < len ; ii++)
if (isupper(string[ii]))
count++;
return count;
}
Here is the foreign function definition using :reference-pass:
(fli:define-foreign-function (count-upper "count_upper" :source)
((string (:reference-pass :ef-mb-string)))
:result-type :int
:language :c
:calling-convention :cdecl)
(count-upper "ABCdef")
=>
3
In this example no Lisp string is needed when calling. The :reference-return type converts a foreign string of lowercase ASCII characters to a Lisp string on return. Here is the C code for the example.
#include <string.h>
#include <stdlib.h>
__declspec(dllexport) void __cdecl random_string(int length, char *string)
{
int ii;
for (ii = 0; ii < length ; ii++)
string[ii] = 97 + rand() % 26;
string[length] = 0;
}
#include <string.h>
#include <stdlib.h>
void random_string(int length, char *string)
{
int ii;
for (ii = 0; ii < length ; ii++)
string[ii] = 97 + rand() % 26;
string[length] = 0;
}
In this foreign function definition the :reference-return type must specify a size, since memory is allocated for it before calling the C function. Note also the use of :lambda-list so that the caller does not have to pass a dummy argument for the returned string, and :result-type nil corresponding to the void declaration of the C function.
(fli:define-foreign-function (random-string
"random_string"
:source)
((length :int)
(return-string (:reference-return
(:ef-mb-string
:limit 256))))
:result-type nil
:lambda-list (length &aux return-string)
:calling-convention :cdecl)
(random-string 3)
=>
"uxw"
(random-string 6)
=>
"fnfozv"
Here is the C code for the example. On return, the argument string has been modified (the code assumes there is enough space after the string for the extra characters).
#include <stdio.h>
#include <string.h>
__declspec(dllexport) void __cdecl modify(char *string) {
char temp[256];
sprintf(temp, "'%s' modified in a C function", string);
strcpy(string, temp);
}
#include <stdio.h>
#include <string.h>
void modify(char *string) {
char temp[256];
sprintf(temp, "'%s' modified in a C function", string);
strcpy(string, temp);
}
Here are three approaches to calling modify from Lisp:
1. Use a fixed size buffer in define-foreign-function. This uses the :reference type, which automatically allocates a temporary foreign object, fills it with data converted from the Lisp object, passes a pointer to C and converts the data in the foreign object back into a new Lisp object on return. Note that the Lisp object passed to the function is not modified. This is the neatest way, provided you can bound the size of the result string at compile-time.
(fli:define-foreign-function (dff-modify "modify" :source)
((string (:reference (:ef-mb-string :limit 256))))
:calling-convention :cdecl)
(dff-modify "Lisp String")
=>
"'Lisp String' modified in a C function"
2. Use a fixed size buffer from with-dynamic-foreign-objects. In this case, we do most of the conversion explicitly and define the foreign function as taking a :pointer argument. This is a good approach if you don't know the maximum length when the function is defined, but will know it at compile-time for each call to the function.
(fli:define-foreign-function (wdfo-modify "modify" :source)
((string :pointer))
:calling-convention :cdecl)
(fli:with-dynamic-foreign-objects
((c-string (:ef-mb-string :limit 256)
:initial-element "Lisp String"))
(wdfo-modify c-string)
(fli:convert-from-foreign-string c-string))
=>
"'Lisp String' modified in a C function"
3. With a variable size buffer from allocate-dynamic-foreign-object. In this case, we do all of the conversion explicitly because we need to make an array of the right size, which is only known after the foreign string has been created (the extra 100 bytes are to allow for what the C function inserts into the string). Note that, in order to support arbitrary external formats, the code makes no assumptions about the length of the temporary array being the same as the length of the Lisp string: it does the conversion first using with-foreign-string, which works out the required number of bytes. The use of with-dynamic-foreign-objects provides a dynamic scope for call to allocate-dynamic-foreign-object - on exit, the foreign object will be freed automatically.
(fli:with-foreign-string (temp element-count byte-count)
"Lisp String"
(fli:with-dynamic-foreign-objects ()
(let ((c-string (fli:allocate-dynamic-foreign-object
:type '(:unsigned :byte)
:nelems (+ byte-count 100))))
(fli:replace-foreign-object c-string temp :nelems byte-count)
(wdfo-modify c-string)
(fli:convert-from-foreign-string c-string))))
Suppose you have a C function declared like this:
extern "C" void foo( const char** StringArray);
To call this from Lisp you need to first allocate the foreign memory for each piece of data, that is the array itself and each string. Assuming that foo does not capture any of the pointers, you can give this memory dynamic extent as follows:
(defun convert-to-dynamic-foreign-array (strings)
(let* ((count (length strings))
(array
(fli:allocate-foreign-object
:nelems (1+ count) ; assume NULL terminated
:type '(:pointer :char))))
(dotimes (index count)
(setf (fli:dereference array :index index)
(fli:convert-to-dynamic-foreign-string
(elt strings index))))
(setf (fli:dereference array :index count) nil)
array))
(fli:define-foreign-function (%foo foo)
((string-array (:pointer (:pointer :char)))))
(defun foo (strings)
(fli:with-dynamic-foreign-objects () ; provide a dynamic scope
(%foo (convert-to-dynamic-foreign-array strings))))
Here is a similar example converting Lisp strings to **char or *char[] which by default allocates using malloc (the value :static for the allocation argument):
(defun convert-strings-to-foreign-array (strings &key
(allocation :static))
(let* ((count (length strings))
(array (fli:allocate-foreign-object
:type '(:pointer (:unsigned :char))
:nelems (1+ count)
:initial-element nil
:allocation allocation)))
(loop for index from 0
for string in strings
do (setf (fli:dereference array :index index)
(fli:convert-to-foreign-string
string
:external-format :utf-8
:allocation allocation)))
array))
If you call it frequently, then you will probably want to free the array (and the strings inside it). Alternatively, you can give the array and its strings dynamic scope if the foreign side does not keep a pointer to the data, like this:
(fli:with-dynamic-foreign-objects ()
(let ((array (convert-strings-to-foreign-array
strings :allocation :dynamic)))
(%foo array)))
The :ef-mb-string type is capable of converting between the internal encoding of LispWorks strings (Unicode) and various encodings that may be expected by the foreign code. The encoding on the foreign side is specified by the :external-format argument, which takes an External Format specification.. See the
LispWorks User Guide and Reference Manual
for a more detailed description of external formats.
Consider a variant of the last example where the returned string contains characters beyond the ASCII range.
#include <string.h>
#include <stdlib.h>
__declspec(dllexport) void __cdecl random_string2(int length, char *string)
{
int ii;
for (ii = 0; ii < length ; ii++)
string[ii] = 225 + rand() % 26;
string[length] = 0;
}
#include <string.h>
#include <stdlib.h>
void random_string2(int length, char *string)
{
int ii;
for (ii = 0; ii < length ; ii++)
string[ii] = 225 + rand() % 26;
string[length] = 0;
}
A foreign function defined like random-string above is inadequate by itself here because the default external format is that for the default C locale, ASCII. This will signal error when it encounters a non-ASCII character code. There are two approaches to handling non-ASCII characters.
1. Pass an appropriate external format, in this case it is Latin-1:
(fli:define-foreign-function (random-string2
"random_string2"
:source)
((length :int)
(return-string (:reference-return
(:ef-mb-string
:limit 256
:external-format :latin-1))))
:result-type nil
:lambda-list (length &aux return-string)
:calling-convention :cdecl)
(random-string2 3)
=>
"òãö"
(random-string2 6)
=>
"óãøççâ"
2. Set the locale, using set-locale. This sets the C locale and switches the FLI to use an appropriate default wherever an external-format argument is accepted.
(fli:define-foreign-function (random-string
"random_string2"
:source)
((length :int)
(return-string (:reference-return
(:ef-mb-string
:limit 256))))
:result-type nil
:lambda-list (length &aux return-string)
:calling-convention :cdecl)
On a Windows system with current Code Page for Western European languages:
(fli:set-locale)
=>
(win32:code-page :id 1252)
On a Non-Windows system with a Latin-1/ISO8859-1 default locale:
(fli:set-locale)
=>
:latin-1
After the default external-format has been switched:
(random-string 6)
=>
"≤èñçèõ"
If you do not actually wish to set the C locale, you can call set-locale-encodings which merely switches the FLI to use the specified external formats where an external-format argument is accepted.
You can specify the line terminator in foreign string conversions via the :eol-style parameter in the external-format argument.
By default foreign strings are assumed to have lines terminated according to platform conventions: Linefeed on Non-Windows systems, and Carriage-Return followed by Linefeed on Windows. That is, eol-style defaults to :lf and :crlf respectively. This means that unless you take care to specify the external format :eol-style parameter, you may get unexpected string length when returning a Lisp string.
Consider the following C code example on Windows:
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
__declspec(dllexport) int __cdecl crlf_string(int length, char *string)
{
int ii;
int jj;
for (ii = 0; ii < length ; ii++)
if (ii % 3 == 1) {
string[ii] = 10;
printf("%d\n", ii);
} else
if ((ii > 0) && (ii % 3 == 0)) {
string[ii] = 13;
printf("%d\n", ii);
} else
if (ii % 3 == 2) {
string[ii] = 97 + rand() % 26 ;
printf("%d\n", ii);
}
string[length] = 0;
return length;
}
Call this C function from Lisp:
(fli:define-foreign-function (crlf-string
"crlf_string"
:source)
((length :int)
(return-string (:reference-return
(:ef-mb-string
:limit 256
:external-format :latin-1))))
:lambda-list (length &aux return-string)
:calling-convention :cdecl
:result-type :int)
(multiple-value-bind (length string)
(crlf-string 99)
(format t "~&C length ~D, Lisp string length ~D~%" length (length string)))
=>
C length 99, Lisp string length 67
Each two character CR LF sequence in the foreign string has been mapped to a single LF character in the Lisp string. If you want to return a Lisp string and not do line terminator conversion, then you must specify the eol-style as in this example:
(fli:define-foreign-function (crlf-string
"crlf_string"
:source)
((length :int)
(return-string (:reference-return
(:ef-mb-string
:limit 256
:external-format (:latin-1 :eol-style :lf)))))
:lambda-list (length &aux return-string)
:calling-convention :cdecl
:result-type :int)
(multiple-value-bind (length string)
(crlf-string 99)
(format t "~&C length ~D, Lisp string length ~D~%" length (length string)))
=>
C length 99, Lisp string length 99
Functions in the Win32 API that handle strings come in two flavors, one for ANSI strings and one for Unicode strings. Supported versions of Microsoft Windows support both flavors. The functions are named with a single letter suffix, an A for the ANSI functions and a W for the Unicode functions. So for example both CreateFileA and CreateFileW exist. In C, this is finessed by the use of #define in the header files.
There are three ways to handle this:
(define-foreign-function (create-file "CreateFileA")
((lpFileName win32:lpcstr) ...))
This will prevent the use of Unicode strings but this is typically only a problem if you are handling mixed language data. Be sure to use the correct FLI types win32:str, win32:lpcstr and so on when explicitly interfacing to an ANSI Win32 function.
W function explicitly, for example:(define-foreign-function (create-file "CreateFileW")
((lpFileName win32:lpcwstr) ...))
This will allow use of Unicode strings. Be sure to use the correct FLI types win32:wstr, win32:lpcwstr and so on when explicitly interfacing to a Unicode Win32 function.
:dbcs in define-foreign-function and omit the single letter suffix, for example:(fli:define-foreign-function (create-file "CreateFile" :dbcs)
((lpFileName win32:lpctstr) ...))
This will cause it to use the Unicode W function implicitly in supported versions of Windows. (In some older operating systems such as Windows ME, this mechanism would implicitly use the ANSI A function.)
In all cases, as well as calling the correct function, you must encode/decode any string arguments and results correctly, to match the A or W in the function name. The foreign types win32:tstr, win32:lpctstr and win32:lptstr automatically switch between ANSI and Unicode strings and correspond to the typical ones found in the Win32 API. For more information about these foreign types, see their manual pages in the
LispWorks User Guide and Reference Manual
.
If you wish a string argument to accept nil and pass it as a null pointer, or to return a null pointer as Lisp value nil, use the :allow-null argument to the :reference types.
The C function strcap in the following example modifies a string, but also accepts and returns a null pointer if passed.
#include <string.h>
#include <ctype.h>
__declspec(dllexport) void __cdecl strcap(char *string)
{
int len;
int ii;
if (string) {
len = strlen(string);
if (len > 0) {
for (ii = len - 1; ii > 0; ii--)
if (isupper(string[ii]))
string[ii] = tolower(string[ii]);
if (islower(string[0]))
string[0] = toupper(string[0]);
}
}
}
#include <string.h>
#include <ctype.h>
void strcap(char *string)
{
int len;
int ii;
if (string) {
len = strlen(string);
if (len > 0) {
for (ii = len - 1; ii > 0; ii--)
if (isupper(string[ii]))
string[ii] = tolower(string[ii]);
if (islower(string[0]))
string[0] = toupper(string[0]);
}
}
}
With this following foreign function definition:
(fli:define-foreign-function (strcap "strcap" :source)
((string (:reference :ef-mb-string)))
:language
:c
:calling-convention
:cdecl)
(strcap "abC")
=>
"Abc"
However (strcap nil) signals error because the :ef-mb-string type expects a string.
Using :allow-null allows nil to be passed:
(fli:define-foreign-function (strcap "strcap" :source)
((string (:reference :ef-mb-string :allow-null t)))
:language
:c
:calling-convention
:cdecl)
(strcap nil)
=>
nil
Note that with-foreign-string, convert-to-foreign-string and convert-from-foreign-string also accept an :allow-null argument. So another way to call strcap and allow the null pointer is:
(fli:define-foreign-function (strcap "strcap" :source)
((string :pointer))
:language
:c
:calling-convention
:cdecl)
(defun c-string-capitalize (string)
(fli:with-foreign-string (ptr elts bytes :allow-null t)
string
(declare (ignore elts bytes))
(strcap ptr)
(fli:convert-from-foreign-string ptr :allow-null t)))
(c-string-capitalize "abC")
=>
"Abc"
(c-string-capitalize nil)
=>
nil
LispWorks Foreign Language Interface User Guide and Reference Manual - 29 Sep 2017