Due to the way CMU Common Lisp manages memory, the amount of memory that can
be dynamically allocated by malloc or (page )make-alien is
limited8.1.
To overcome this limitation, it is possible to access the content of Lisp arrays which are limited only by the amount of physical memory and swap space available. However, this technique is only useful if the foreign function takes pointers to memory instead of allocating memory for itself. In latter case, you will have to modify the foreign functions.
This technique takes advantage of the fact that CMU Common Lisp has specialized array types (see section specialized-array-types) that match a typical C array. For example, a (simple-array double-float (100)) is stored in memory in essentially the same way as the C array double x[100] would be. The following function allows us to get the physical address of such a Lisp array:
ARRAY must be a specialized array type in CMU Lisp. This means ARRAY must be an array of one of the following types:
double-float single-float (unsigned-byte 32) (unsigned-byte 16) (unsigned-byte 8) (signed-byte 32) (signed-byte 16) (signed-byte 8) " (declare (type (or #+signed-array (array (signed-byte 8)) #+signed-array (array (signed-byte 16)) #+signed-array (array (signed-byte 32)) (array (unsigned-byte 8)) (array (unsigned-byte 16)) (array (unsigned-byte 32)) (array single-float) (array double-float)) array) (optimize (speed 3) (safety 0)) (ext:optimize-interface (safety 3))) ;; with-array-data will get us to the actual data. However, because ;; the array could have been displaced, we need to know where the ;; data starts. (lisp::with-array-data ((data array) (start) (end)) (declare (ignore end)) ;; DATA is a specialized simple-array. Memory is laid out like this: ;; ;; byte offset Value ;; 0 type code (should be 70 for double-float vector) ;; 4 4 * number of elements in vector ;; 8 1st element of vector ;; ... ... ;; (let ((addr (+ 8 (logandc1 7 (kernel:get-lisp-obj-address data)))) (type-size (let ((type (array-element-type data))) (cond ((or (equal type '(signed-byte 8)) (equal type '(unsigned-byte 8))) 1) ((or (equal type '(signed-byte 16)) (equal type '(unsigned-byte 16))) 2) ((or (equal type '(signed-byte 32)) (equal type '(unsigned-byte 32))) 4) ((equal type 'single-float) 4) ((equal type 'double-float) 8) (t (error "Unknown specialized array element type")))))) (declare (type (unsigned-byte 32) addr) (optimize (speed 3) (safety 0) (ext:inhibit-warnings 3))) (system:int-sap (the (unsigned-byte 32) (+ addr (* type-size start)))))))
Assume we have the C function below that we wish to use:
for (k = 0; k < n; ++k) { sum += x[k] * y[k]; } }
(let ((x (make-array 1000000 :element-type 'double-float)) (y (make-array 1000000 :element-type 'double-float))) ;; Initialize X and Y somehow (let ((x-addr (system:int-sap (array-data-address x))) (y-addr (system:int-sap (array-data-address y)))) (dotprod x-addr y-addr 1000000)))