-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make MAGICL Lisp-only by default, introduce MAGICL/EXT-* systems #124
Comments
Broadly speaking I think this is a good idea. Two quick remarks
|
Something like:
Then as a user I can |
FWIW, I'm in favor of completely decoupling the BLAS/LAPACK bindings from the rest of MAGICL. People who want to use just that should not feel like they have to buy into the whole MAGICL API. I'm surprised about the pain points you report. I'd expect most users to have some set of linear algebra libraries in their system. This is certainly true for Mac OS, which bundles the Accelerate framework. Typical Linux users interested in quantum computing will likely already have NumPy and SciPy installed, which have BLAS/LAPACK as prerequisites, usually satisfied via ATLAS (comes with vanilla distributions), Intel MKL (bundled with Conda), or OpenBLAS (for the discerning users out there). As for GNU Fortran, my recollection is that it's only necessary for the (minority of) users that want to rebuild the interfaces from scratch. By the way, why write the linear algebra routines in Lisp by hand? F2CL already bundles many of them. To summarize, I believe the root problem might be software configuration/distribution and, in that case, it would be better to address that directly. The effort spent on reference implementations of algorithms, etc. could be channeled into making MAGICL / Common Lisp more competitive as a language for numerical computing and experimentation (e.g., supporting CUDA, interfacing to PyTorch, etc.). Just my $ |
Can we close this now that #125 merged? |
One of the biggest pain points of MAGICL is having to install LAPACK and GFORTRAN and etc. It's transitively difficult to use projects that depend on MAGICL as well, like quilc.
I propose that we split MAGICL into a pure-Lisp and non-Lisp-acceleerated double system. Specifically, I think there should be the following rough divisions:
Make the
MAGICL
system Lisp-onlyWe have a high-level framework for representing tensors, thanks to @colescott. The rudimentary linear algebra operations are also not too difficult to implement in Lisp. These would be:
We don't need to have mega high-performance implementations in Lisp, just enough to make the base library useful.
Critically, however, we remove all LAPACK stuff from the
MAGICL
system, and shovel it somewhere else as described next...Put non-Lisp bindings into a new system called
MAGICL/EXT-XXX
Specifically, I imagine
MAGICL/EXT-BLAS
,MAGICL/EXT-LAPACK
, andMAGICL/EXT-EXPOKIT
. The/
is used because ASDF understands it as a "sub-system" of sorts, andEXT-
stands for "extension".These systems would contain two main attractions:
MAGICL
packageBy point 2, I mean that generic functions that were responsible for some operations can have methods that are defined or even re-defined by loading this system, thus "accelerating" the pure-Lisp-based routines above with Fortran/C-written ones.
One open question here is: How is new functionality introduced? By way of a new package? Or exporting new symbols from
MAGICL
? One possibility is that the generic functions are defined in MAGICL, but no methods are adorned. (Maybe a default "not implemented" error or something is specialized on T.)Create a
MAGICL/FANCY
meta-packageI don't actually know what to call it, but
MAGICL/FANCY
is a dummy system that depends on all of theMAGICL/EXT-XXX
extension systems. If you want a souped up MAGICL, loadMAGICL/FANCY
.Add a
*PREFER-LISP*
variableThis is optional, but it would be nice if one could prefer Lisp implementations of functions when possible, and
*PREFER-LISP*
could control that.So now what?
I'd love comments and thoughts on the proposal. MAGICL development has been slow and steady, and having worked with it for a while, I think the above represent a good next step.
The text was updated successfully, but these errors were encountered: