Matrix Algebra proGrams In Common Lisp
Clone or download
Latest commit 472cf75 Oct 18, 2018
Permalink
Failed to load latest commit information.
doc remove hard libgfortran dependency Jun 14, 2018
docker Move Dockerfiles into their own subdirectory, add convenience Makefile Jan 26, 2018
tests move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
transcendental move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
LICENSE-expokit.txt include the expokit license Jan 26, 2018
LICENSE.txt clean up and rename to MAGICL Jul 29, 2017
README.md move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
VERSION.txt bump VERSION.txt Oct 18, 2018
blas-cffi.lisp regenerate all files Oct 2, 2017
cffi-types.lisp excise FNV and use lisp vectors Sep 23, 2017
examples.lisp decommission &rest parameters for matrix construction Oct 27, 2017
expokit-cffi.lisp move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
generate-interface.lisp move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
high-level.lisp move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
lapack-cffi.lisp regenerate all files Oct 2, 2017
load-libs.lisp prefer brew BLAS over system Oct 18, 2018
magicl-examples.asd clean up and rename to MAGICL Jul 29, 2017
magicl-gen.asd move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
magicl-tests.asd move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
magicl-transcendental.asd move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
magicl.asd move expokit functions to MAGICL-TRANSCENDENTAL Aug 7, 2018
magicl.lisp initial ECL support Sep 24, 2017
packages.lisp add DAGGER and DIRECT-SUM Aug 6, 2018
random.lisp decommission &rest parameters for matrix construction Oct 27, 2017
reify.lisp MATRIX reification on core saving Sep 15, 2017
with-array-pointers.lisp pin array storage vector, not array itself, in SBCL Apr 3, 2018

README.md

MAGICL

Build Status

Matrix Algebra proGrams In Common Lisp by Rigetti Computing. (née FLAIL: Finally, Linear Algebra In Lisp!)

Load it with (ql:quickload :magicl).

Test it with (asdf:test-system :magicl).

(Note: The high-level interface is experimental and subject to rapid change.)

Requirements

  • SBCL (> 1.3.19) or CCL (>= 1.11) on AMD64
  • quicklisp
  • libffi
  • BLAS and LAPACK

A separate system, magic-transcendental, contains transcendental functions, and requires another dependency called Expokit.

Detailed instructions on how to install libffi, BLAS/LAPACK, and Expokit can be found here.

Currently this library is SBCL- and CCL-only. The non-portable code is in with-array-pointers.lisp and magicl.lisp.

Testing MAGICL

You can run the MAGICL tests from your Lisp REPL with:

(ql:quickload :magicl-tests)
(asdf:test-system :magicl)

Note that the tests use both magicl and magicl-transcendental and hence requires Expokit.

This repository is also set up with Sempahore CI, and uses Docker for building MAGICL and running its tests. You can emulate Semaphore's behavior by using the following commands from the top-level directory:

make -C docker
make -C docker test

Note, doing this requires that you have docker installed on your machine.

Showing Available Functions

Some distributions of a library don't actually provide all of the functions of the reference BLAS and LAPACK. One can look at a summary of available and unavailable functions with the function magicl:print-availability-report. By default, it will show all functions and their availability. There are three arguments to fine-tune this behavior:

  1. Key :show-available <boolean> (default t): show available functions
  2. Key :show-unavailable <boolean> (default t): show unavailable functions
  3. Key :search <string>: only show functions which have <string> as a substring. This argument takes into account the previous two arguments.

For example, we can look for all available functions which might relate to svd by doing the following:

CL-USER> (magicl:print-availability-report :search "svd" :show-unavailable nil)
        Fortran Function        Lisp Function
------------------------------------------------------------------------

Library LIBBLAS: /usr/local/opt/lapack/lib/libblas.dylib

Library LIBLAPACK: /usr/local/opt/lapack/lib/liblapack.dylib
    [x] CGESVD                  MAGICL.LAPACK-CFFI:%CGESVD
    [x] CGESVDX                 MAGICL.LAPACK-CFFI:%CGESVDX
    [x] CGGSVD3                 MAGICL.LAPACK-CFFI:%CGGSVD3
    [x] DBDSVDX                 MAGICL.LAPACK-CFFI:%DBDSVDX
    [x] DGESVD                  MAGICL.LAPACK-CFFI:%DGESVD
    [x] DGESVDX                 MAGICL.LAPACK-CFFI:%DGESVDX
    [x] DGGSVD3                 MAGICL.LAPACK-CFFI:%DGGSVD3
    [x] SBDSVDX                 MAGICL.LAPACK-CFFI:%SBDSVDX
    [x] SGESVD                  MAGICL.LAPACK-CFFI:%SGESVD
    [x] SGESVDX                 MAGICL.LAPACK-CFFI:%SGESVDX
    [x] SGGSVD3                 MAGICL.LAPACK-CFFI:%SGGSVD3
    [x] ZGESVD                  MAGICL.LAPACK-CFFI:%ZGESVD
    [x] ZGESVDX                 MAGICL.LAPACK-CFFI:%ZGESVDX
    [x] ZGGSVD3                 MAGICL.LAPACK-CFFI:%ZGGSVD3

Generating BLAS and LAPACK Bindings

This library takes the approach of automatically generating the bindings to BLAS, LAPACK, and Expokit without relying on any special tools.

In order to generate the bindings, you will need to download the Fortran 90 source tarballs for BLAS/LAPACK and Expokit. Once downloaded, extract the tarballs into a directory and re-generate the bindings with the following commands:

(ql:quickload :magicl-gen)
(in-package :magicl.generate-interface)
(generate-blapack-files #P"/path/to/lapack-3.7.1/")
(generate-expokit-files #P"/path/to/expokit/")

Currently this will write to the source distribution directory of MAGICL, namely the files blas-cffi.lisp, lapack-cffi.lisp, and expokit-cffi.lisp.

History and Credits

MAGICL development started at Rigetti Computing by Robert Smith and Joe Lin in 2017.

CL-BLAPACK is a library developed by Ryan Rifkin and Evan Monroig. Rigetti Computing created a fork of this library and renamed it MAGICL, and made significant changes that departed from the original design, including:

  • Fixing several bugs in the Fortran parsing to make it work with the latest reference BLAS and LAPACK, leading to significant refactoring.
  • Adding support for matrix exponentiation with Expokit.
  • Adding support for loading various BLAS and LAPACK implementations.
  • Removing the use of the FNV library in favor of native Lisp arrays.
  • Adding a high-level interface to various functions.
  • Adding function availability reporting.

The most important common design decision between CL-BLAPACK and MAGICL is allowing direct access to the Fortran library functions by way of automatically generated Lisp bindings from the reference sources.