ENH: allow using symbol-suffixed 64-bit BLAS/LAPACK for numpy.dot and linalg #15012
Conversation
This is an emerging "standard" for 64-bit BLAS/LAPACK, avoiding symbol clashes with 32-bit BLAS/LAPACK, originally introduced for Julia. OpenBLAS can be compiled with 64-bit integer size and symbol suffix '64_' (INTERFACE64=1 SYMBOLSUFFIX=64_). OpenBLAS built with this setting is also provided by some Linux distributions (e.g. Fedora's 64-bit openblas packages).
This is enabled by setting NPY_USE_BLAS64_=1 environment variable at build time. If set, Numpy will be linked both against a 32-bit BLAS and a symbol-suffixed 64-bit BLAS. (This is safe and does not cause symbol clashes thanks to the suffixing.)
…able When Numpy is compiled with BLAS64_ enabled, use the 64-bit CBLAS routines to provide cblasfuncs, and dot() et al.
pv
changed the title
|
I checked with OpenBLAS and this works. I also installed a scipy with MKL and replaced |
| @@ -10,10 +10,20 @@ | |||
| #include <assert.h> | |||
| #include <numpy/arrayobject.h> | |||
| #include "npy_cblas.h" | |||
| #include "npy_cblas64_.h" | |||
There was a problem hiding this comment.
To match the symbol name convention.
|
This is really useful for people with lots of memory. We should add checks so that calling BLAS with a large array does not segfault as in this example, maybe as a separate issue since this PR is quite large as it is. Name bikeshedding: is the |
|
There's of course no real standard suffix, but |
|
I'll add the test in a separate PR, as we'll probably need some machinery to stop it from running on low-memory systems, which it would trash to standstill... |
|
There is a test for large zip files that tries to allocate and skips if it cannot, with a |
|
The "user experience" with not checking beforehand is sometimes bad (i.e. machine hangs). gh-15021 has a more elaborate suggestion for the low-memory check. |
|
Azure failures seem spurious ("[NuGet] The operation has timed out" when trying to install mingw). |
|
Thanks Pauli. It will be good to get this out there. If nothing else it will help set the standard for the 64 bit move. |
|
Should this have a release note? |
|
Probably yes. I forgot numpy is using towncrier, sorry. Will have time for this onlt later this week, so if someone beats me to it, ok.
I also looked at getting matthew's manylinux/etc openblas builds produce 64_ variant, cf. pv/openblas-libs@blas64_ -- quite fiddly to deal with the cis, but that's basically finished now, even if didn't have time to actually test the product vs numpy. Anyway, getting that merged to production is probably required before eg. adding blas64 travis target.
…On December 2, 2019 5:56:33 PM UTC, Eric Wieser ***@***.***> wrote:
Should this have a release note?
|
|
The big picture being that 64bit pypi wheels could be eventually switched to use 64bit blas, of course. But I guess this should cook for some time in master first and maybe see some more serious use.
…On December 2, 2019 7:40:25 PM UTC, Pauli Virtanen ***@***.***> wrote:
Probably yes. I forgot numpy is using towncrier, sorry. Will have time
for this onlt later this week, so if someone beats me to it, ok.
I also looked at getting matthew's manylinux/etc openblas builds
produce 64_ variant, cf. ***@***.***_ -- quite fiddly to
deal with the cis, but that's basically finished now, even if didn't
have time to actually test the product vs numpy. Anyway, getting that
merged to production is probably required before eg. adding blas64
travis target.
On December 2, 2019 5:56:33 PM UTC, Eric Wieser
***@***.***> wrote:
>Should this have a release note?
|
|
Really nice to see numpy is becoming more 64bit and all the great efforts toward this! |
|
@refraction-ray, to do that, you'll have to create a shared library from MKL static library exporting only the BLAS, LAPACK symbols and then renaming those symbols. |
|
@isuruf by renaming, you mean using GNU objcopy to rename all symbol names following the new 64bit prefix convention openblas and numpy utilized? And I am not sure about only export BLAS LAPACK symbols part. What is the danger if all symbols are exported to the new .so file and how to fast specify all BLAS and LAPACK symbols to be exported? |
yes.
There are symbols already in the MKL libs with and without
Use a linker script. |
Changing these to support ILP64 blas was missed in numpygh-15012
Changing these to support ILP64 blas was missed in numpygh-15012
(This is partially for discussion, but should be functional:)
Add support for 64-bit BLAS/LAPACK with
64_symbol suffix (henceforth "BLAS64_").BLAS64_ is 64-bit BLAS with 64-bit integer interface (ILP64) with
64_appended to all symbol names. This what you get withINTERFACE64=1 SYMBOLSUFFIX=64_in OpenBLAS, and is the approach used by Julia. Fedora also has 64-bit OpenBLAS packages with this configuration. The point with this is that you can safely link both to 32-bit and 64-bit BLAS at the same time, without having to coordinate all packages. In particular, numpy can link to BLAS64_ and you don't get segfaults if you doimport numpyfrom a program that also links to 32-bit BLAS (which would be the case with 64-bit BLAS/LAPACK if done in the usual way --- for ELF it mostly problem for programs embedding Python, not so much for standalone Python, but Numpy probably cares about both use cases and the 'usual' way is no-go).See OpenMathLib/OpenBLAS#646 and JuliaLang/julia#4923 (and also #13956 , #5906 )
This PR adds numpy.distutils support for detecting OpenBLAS64_, based on the library name and checking it actually contains the correct symbols.
It adds a build-time environment variable
NPY_USE_BLAS64_=1. If it's set, Numpy gets linked only with 64-bit BLAS. It's off by default for now, but it should be safe to turn it to be on by default later on.The 64-bit BLAS functions are then used to implement
cblasfuncs.c, so that numpy.dot et al. get the 64-bit goodness. Moreover, the latter commits also implement its usage in numpy.linalg.Addresses: #5533 (if you have the right 64-bit BLAS installed). Moreover, the numpy.distutils support should be useful for all downstream libraries.
(I don't have a machine with enough memory to check this actually fixes the issues with
n > 2**32-1matrix products, but the call does end up inlibopenblas64_.soso it should.)