next up previous contents index
Next: Step-by-Step Alien Example Up: Alien Function Calls Previous: Calling Lisp from C   Contents   Index

Accessing Lisp Arrays

Due to the way CMU Common Lisp manages memory, the amount of memory that can be dynamically allocated by malloc or (page [*])make-alien is limited8.1.

To overcome this limitation, it is possible to access the content of Lisp arrays which are limited only by the amount of physical memory and swap space available. However, this technique is only useful if the foreign function takes pointers to memory instead of allocating memory for itself. In latter case, you will have to modify the foreign functions.

This technique takes advantage of the fact that CMU Common Lisp has specialized array types (see section specialized-array-types) that match a typical C array. For example, a (simple-array double-float (100)) is stored in memory in essentially the same way as the C array double x[100] would be. The following function allows us to get the physical address of such a Lisp array:

(defun array-data-address (array) "Return the physical address of where the actual data of an array is stored.

ARRAY must be a specialized array type in CMU Lisp.  This means ARRAY must be an array of one of the following types:

double-float single-float (unsigned-byte 32) (unsigned-byte 16) (unsigned-byte  8) (signed-byte 32) (signed-byte 16) (signed-byte  8) " (declare (type (or #+signed-array (array (signed-byte 8)) #+signed-array (array (signed-byte 16)) #+signed-array (array (signed-byte 32)) (array (unsigned-byte 8)) (array (unsigned-byte 16)) (array (unsigned-byte 32)) (array single-float) (array double-float)) array) (optimize (speed 3) (safety 0)) (ext:optimize-interface (safety 3))) ;; with-array-data will get us to the actual data.  However, because ;; the array could have been displaced, we need to know where the ;; data starts. (lisp::with-array-data ((data array) (start) (end)) (declare (ignore end)) ;; DATA is a specialized simple-array.  Memory is laid out like this: ;; ;;   byte offset    Value ;;        0         type code (should be 70 for double-float vector) ;;        4         4 * number of elements in vector ;;        8         1st element of vector ;;      ...         ... ;; (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data)))) (type-size (let ((type (array-element-type data))) (cond ((or (equal type '(signed-byte 8)) (equal type '(unsigned-byte 8))) 1) ((or (equal type '(signed-byte 16)) (equal type '(unsigned-byte 16))) 2) ((or (equal type '(signed-byte 32)) (equal type '(unsigned-byte 32))) 4) ((equal type 'single-float) 4) ((equal type 'double-float) 8) (t (error "Unknown specialized array element type")))))) (declare (type (unsigned-byte 32) addr) (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3))) (system:int-sap (the (unsigned-byte 32) (+ addr (* type-size start)))))))

Assume we have the C function below that we wish to use:

double dotprod(double* x, double* y, int n) { int k; double sum = 0;

for (k = 0; k < n; ++k) { sum += x[k] * y[k]; } }

The following example generates two large arrays in Lisp, and calls the C function to do the desired computation. This would not have been possible using malloc or make-alien since we need about 16 MB of memory to hold the two arrays.
(def-alien-routine "dotprod" double (x (* double-float) :in) (y (* double-float) :in) (n int :in))

(let ((x (make-array 1000000 :element-type 'double-float)) (y (make-array 1000000 :element-type 'double-float))) ;; Initialize X and Y somehow (let ((x-addr (system:int-sap (array-data-address x))) (y-addr (system:int-sap (array-data-address y)))) (dotprod x-addr y-addr 1000000)))    

In this example, it may be useful to wrap the inner let expression in an unwind-protect that first turns off garbage collection and then turns garbage collection on afterwards. This will prevent garbage collection from moving x and y after we have obtained the (now erroneous) addresses but before the call todotprod is made.


next up previous contents index
Next: Step-by-Step Alien Example Up: Alien Function Calls Previous: Calling Lisp from C   Contents   Index
Peter Van Eynde 2000-02-08