Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Impossible to use ACML with system installed ATLAS #2728

Closed
bfroehle opened this Issue Nov 13, 2012 · 8 comments

Comments

Projects
None yet
2 participants
Contributor

bfroehle commented Nov 13, 2012

As far as I can tell it's impossible to configure site.cfg to link to ACML (+CBLAS) with a system installed ATLAS, using maintenance/1.6.x. (I haven't tried with newer versions).

For example, site.cfg would have me believe that I should be able to link to ACML with:

[blas_opt]
libraries = cblas,acml
library_dirs = /opt/acml5.2.0/gfortran64_fma4/lib

[lapack_opt]
libraries = acml
library_dirs = /opt/acml5.2.0/gfortran64_fma4/lib

But running python setup.py build instead links to the system installed ATLAS:

$ ldd build/lib.linux-x86_64-2.7/numpy/core/_dotblas.so
    linux-vdso.so.1 =>  (0x00007fffbdf80000)
    libptf77blas.so.3 => /usr/lib64/atlas/libptf77blas.so.3 (0x00007f685ed10000)
    libptcblas.so.3 => /usr/lib64/atlas/libptcblas.so.3 (0x00007f685eaf0000)
    libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f685e490000)
...

This is easily seen as well by running get_info('blas_opt'):

$ cat test.py
import __builtin__ as builtins
builtins.__NUMPY_SETUP__ = True
import numpy.distutils.system_info as si
print si.get_info('blas_opt')

$ python test.py 
Running from numpy source directory.
...
{'libraries': ['ptf77blas', 'ptcblas', 'atlas'], 'library_dirs': ['/usr/lib64/atlas'], 'define_macros': [('ATLAS_INFO', '"\\"3.8.4\\""')], 'language': 'c', 'include_dirs': ['/usr/include']}

How can I convince the setup process to not ignore my site.cfg file?

Owner

rgommers commented Nov 14, 2012

If you feel like living on the edge, Bento has recently grown the ability to configure BLAS/LAPACK easily: eed2372

I have no idea why distutils is ignoring site.cfg for you.

Contributor

bfroehle commented Nov 14, 2012

Well there are a bunch of reasons. First is that blas_opt_info doesn't set section or dir_env_var so it doesn't seem to be overridable by site.cfg.

Then the detection code explicitly prefers atlas_blas if it can be found. That's what's happening in my case.

I am able to explicitly prevent the script from finding ATLAS, e.g. by adding site.cfg

[atlas]
library_dirs = 

However then the values in [blas_opt] are still ignored (because section isn't set in the blas_opt_info class...).

Owner

rgommers commented Nov 14, 2012

That seems simply incorrect then, site.cfg should override the default. Sounds like you've almost fixed it already, will you open a PR?

Contributor

bfroehle commented Nov 14, 2012

Well, the problem is subtle. Merely setting section = 'blas_opt' in the blas_opt_info class isn't sufficient as then it'll just append the site.cfg values to whatever it auto-detected.

In general it appears that the blas_opt acts as a "meta-section" where is checks for MKL, ATLAS, and then a regular BLAS. Ditto for lapack_opt and fftw_opt, etc.

I'm able to trick the system into thinking that ATLAS is unavailable, e.g. by adding to site.cfg

[atlas]
library_dirs =

However this then get_info('blas_opt') results in:

{'libraries': ['cblas', 'acml'], 'library_dirs': ['/scratch/threedg/acml/5.2.0/gfortran64_fma4/lib'], 'language': 'f77', 'define_macros': [('NO_ATLAS_INFO', 1)]}

Which is fine, except that NO_ATLAS_INFO is has the unfortunate side effect of disabling the compilation of _dotblas.so. This is undesirable as I do have a working copy of cblas to link with.

The documentation of site.cfg led me to believe that the code would instead run as:

  • See if there is a [NAME_opt] section in site.cfg. If so use it.
  • If not, fall back on all of the previous checks.
Contributor

bfroehle commented Nov 14, 2012

In my case I ended up using something like:

class lapack_opt_info(system_info):
    section = 'lapack_opt'
    notfounderror = LapackNotFoundError

    def calc_info(self):

        info = self.calc_libraries_info()
        if info:
            self.set_info(**info)
            return

        ...

I'm happy to open a pull request for discussion if you think that'd be appropriate.

Owner

rgommers commented Nov 15, 2012

Your expectation of what it should do is the same as mine. Opening a PR for discussion sounds good.

@bfroehle bfroehle added a commit to bfroehle/numpy that referenced this issue Nov 18, 2012

@bfroehle bfroehle ENH: Respect blas_opt and lapack_opt groups in site.cfg.
Previously blas_opt and lapack_opt would only be merged into the
autodetected BLAS and LAPACK libraries. This prevented using custom
LAPACK and BLAS implementation if a system installed version of
ATLAS (or similar) was present.

Now the presence of a blas_opt or lapack_opt section completely
overrides the respective autodetection mechanisms, as the site.cfg
documentation would already lead you to believe.

Closes #2728.
781292a
Contributor

bfroehle commented Nov 18, 2012

Thanks for the patience; I have been out of town for several days. I have opened pull request #2751 for discussion.

Owner

rgommers commented Nov 26, 2016

AFAIK the documentation changes to site.cfg have landed. We still don't have explicit support for ACML, but given that AMD announced the end-of-life of ACML (http://developer.amd.com/tools-and-sdks/archive/compute/amd-core-math-library-acml/) I believe we can close this now. There are enough other issues regarding fixing site.cfg behavior to "do what I say".

Thanks @bfroehle.

@rgommers rgommers closed this Nov 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment