New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an easy way to check BLAS/LAPACK linkage from python #3912

Closed
davidosterberg opened this Issue Oct 13, 2013 · 14 comments

Comments

Projects
None yet
7 participants
@davidosterberg

davidosterberg commented Oct 13, 2013

According to the answers to this stackoverflow question the only way to check if Numpy is using the system BLAS/LAPACK is to inspect the output of ldd on some library belonging to numpy hidden deep in the filesystem. In fact I am not 100 % sure that I interprete my results correctly.

It would be nice with something like

>>>print(numpy._lapack_version_)
3.4.2

>>>print(numpy._blas_version_)
OpenBLAS 0.2.8

or if we are using the fallback

>>>print(numpy._lapack_version_)
fallback
@charris

This comment has been minimized.

Show comment
Hide comment
@charris

charris Oct 13, 2013

Member

You can get some information from np.__config__.

In [5]: np.__config__.show()
atlas_threads_info:
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = f77
    include_dirs = ['/usr/include']
blas_opt_info:
    libraries = ['ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = c
    include_dirs = ['/usr/include']
atlas_blas_threads_info:
    libraries = ['ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = c
    include_dirs = ['/usr/include']
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = f77
    include_dirs = ['/usr/include']
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE
Member

charris commented Oct 13, 2013

You can get some information from np.__config__.

In [5]: np.__config__.show()
atlas_threads_info:
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = f77
    include_dirs = ['/usr/include']
blas_opt_info:
    libraries = ['ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = c
    include_dirs = ['/usr/include']
atlas_blas_threads_info:
    libraries = ['ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = c
    include_dirs = ['/usr/include']
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/usr/lib64/atlas']
    define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')]
    language = f77
    include_dirs = ['/usr/include']
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE
@davidosterberg

This comment has been minimized.

Show comment
Hide comment
@davidosterberg

davidosterberg Oct 13, 2013

Thanks!
I wish this was more well known.

davidosterberg commented Oct 13, 2013

Thanks!
I wish this was more well known.

@charris

This comment has been minimized.

Show comment
Hide comment
@charris

charris Oct 13, 2013

Member

I'm not sure what happens when several libraries are available though.

Member

charris commented Oct 13, 2013

I'm not sure what happens when several libraries are available though.

@nouiz

This comment has been minimized.

Show comment
Hide comment
@nouiz

nouiz Oct 15, 2013

Contributor

I'm not sure I understand your question. If you mean there is many library
in the OS with the names gived in by numpy.config? In that case, it use
the normal OS behavior when loading a library: the first one that is found
by the OS is used. But I think you know that, so your question is probably
something else.

On Sun, Oct 13, 2013 at 5:22 PM, Charles Harris notifications@github.comwrote:

I'm not sure what happens when several libraries are available though.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3912#issuecomment-26227470
.

Contributor

nouiz commented Oct 15, 2013

I'm not sure I understand your question. If you mean there is many library
in the OS with the names gived in by numpy.config? In that case, it use
the normal OS behavior when loading a library: the first one that is found
by the OS is used. But I think you know that, so your question is probably
something else.

On Sun, Oct 13, 2013 at 5:22 PM, Charles Harris notifications@github.comwrote:

I'm not sure what happens when several libraries are available though.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3912#issuecomment-26227470
.

@charris

This comment has been minimized.

Show comment
Hide comment
@charris

charris Oct 15, 2013

Member

I think the original question was which library was actually linked, not just detected. Since I don't have multiple library options, I don't know what __config__.show() does in that case.

Member

charris commented Oct 15, 2013

I think the original question was which library was actually linked, not just detected. Since I don't have multiple library options, I don't know what __config__.show() does in that case.

@nouiz

This comment has been minimized.

Show comment
Hide comment
@nouiz

nouiz Oct 15, 2013

Contributor

config.show() indicate what was used during NumPy compilation. We can't
use those name to determine witch implementation of the library used, as
they can be remaned. In Debian, if you install openblas, they have
libblas.so link to it for example.

If you want to know the name used, config.show() give you that. If you
want to know the one used, the only way I know is ldd, as we need to
resolve the one loaded by the OS. I don't know if we can check the one
already loaded in C. But I suppose it could be possible, but not easy.

On Tue, Oct 15, 2013 at 9:20 AM, Charles Harris notifications@github.comwrote:

I think the original question was which library was actually linked, not
just detected. Since I don't have multiple library options, I don't know
what config.show() does in that case.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3912#issuecomment-26333087
.

Contributor

nouiz commented Oct 15, 2013

config.show() indicate what was used during NumPy compilation. We can't
use those name to determine witch implementation of the library used, as
they can be remaned. In Debian, if you install openblas, they have
libblas.so link to it for example.

If you want to know the name used, config.show() give you that. If you
want to know the one used, the only way I know is ldd, as we need to
resolve the one loaded by the OS. I don't know if we can check the one
already loaded in C. But I suppose it could be possible, but not easy.

On Tue, Oct 15, 2013 at 9:20 AM, Charles Harris notifications@github.comwrote:

I think the original question was which library was actually linked, not
just detected. Since I don't have multiple library options, I don't know
what config.show() does in that case.


Reply to this email directly or view it on GitHubhttps://github.com//issues/3912#issuecomment-26333087
.

@davidosterberg

This comment has been minimized.

Show comment
Hide comment
@davidosterberg

davidosterberg Oct 15, 2013

Well, I request a feature that let's the user verify (from python) that Numpy's fallback linear algebra package is not being used. I was happy to learn about __config__.show() as it appear to answer most of my needs in practice. Of course things could have changed since Numpy was compiled but I guess that is not as likely. That's why I closed the issue.

Numpy can be compiled even when the development environment isn't set up correctly. It will then link to its slow fallback library. I think this is what we usually want. I know I was happy about that when I was compiling Numpy on my N900 phone :). But I believe that lot's of people are running the fallback without even realizing it.

davidosterberg commented Oct 15, 2013

Well, I request a feature that let's the user verify (from python) that Numpy's fallback linear algebra package is not being used. I was happy to learn about __config__.show() as it appear to answer most of my needs in practice. Of course things could have changed since Numpy was compiled but I guess that is not as likely. That's why I closed the issue.

Numpy can be compiled even when the development environment isn't set up correctly. It will then link to its slow fallback library. I think this is what we usually want. I know I was happy about that when I was compiling Numpy on my N900 phone :). But I believe that lot's of people are running the fallback without even realizing it.

@stephenshank

This comment has been minimized.

Show comment
Hide comment
@stephenshank

stephenshank Feb 13, 2015

Why do these not provide FFT information? Is this also easily accessible?

stephenshank commented Feb 13, 2015

Why do these not provide FFT information? Is this also easily accessible?

@juliantaylor

This comment has been minimized.

Show comment
Hide comment
@juliantaylor

juliantaylor Feb 13, 2015

Contributor

numpy only has one possible fft choice, which is a copy of fftpack, there are third party numpy compatible wrappers for other libraries available, e.g. pyfftw

Contributor

juliantaylor commented Feb 13, 2015

numpy only has one possible fft choice, which is a copy of fftpack, there are third party numpy compatible wrappers for other libraries available, e.g. pyfftw

@stephenshank

This comment has been minimized.

Show comment
Hide comment
@stephenshank

stephenshank Feb 13, 2015

Thanks for your answer! But I am still confused... what then is the purpose of specifying fftw in numpy/site.cfg?

stephenshank commented Feb 13, 2015

Thanks for your answer! But I am still confused... what then is the purpose of specifying fftw in numpy/site.cfg?

@juliantaylor

This comment has been minimized.

Show comment
Hide comment
@juliantaylor

juliantaylor Feb 13, 2015

Contributor

its for third party libraries using numpy.distutils which can also use the site.cfg. I think scipy used to use the fftw config long ago, but it was unmaintained and removed

Contributor

juliantaylor commented Feb 13, 2015

its for third party libraries using numpy.distutils which can also use the site.cfg. I think scipy used to use the fftw config long ago, but it was unmaintained and removed

@juliantaylor

This comment has been minimized.

Show comment
Hide comment
@juliantaylor

juliantaylor Feb 13, 2015

Contributor

the comment of those keys have only recently been updated to actually say that: 1c11f98

Contributor

juliantaylor commented Feb 13, 2015

the comment of those keys have only recently been updated to actually say that: 1c11f98

@dhimmel

This comment has been minimized.

Show comment
Hide comment
@dhimmel

dhimmel Apr 6, 2018

numpy.show_config() provides the same results as numpy.__config__.show(), but perhaps is a more intended use of the numpy API.

dhimmel commented Apr 6, 2018

numpy.show_config() provides the same results as numpy.__config__.show(), but perhaps is a more intended use of the numpy API.

@sebma

This comment has been minimized.

Show comment
Hide comment
@sebma

sebma Jul 18, 2018

Hi,

I prefer to use np.__config__ because you can then fetch specific information such as :

In [1]: np.__config__.openblas_info
Out[1]:
{'define_macros': [('HAVE_CBLAS', None)],
 'language': 'c',
 'libraries': ['openblas', 'openblas'],
 'library_dirs': ['/usr/local/lib']}

sebma commented Jul 18, 2018

Hi,

I prefer to use np.__config__ because you can then fetch specific information such as :

In [1]: np.__config__.openblas_info
Out[1]:
{'define_macros': [('HAVE_CBLAS', None)],
 'language': 'c',
 'libraries': ['openblas', 'openblas'],
 'library_dirs': ['/usr/local/lib']}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment