Exception when doing - from sklearn import linear_model - in scikit-learn 0.12 on OS X 10.8.2 #1202

Closed
udibr opened this Issue Oct 3, 2012 · 12 comments

7 participants

@udibr

This problem does not happens with scikit-learn 0.11
The problem happens when calling

from cd_fast import sparse_std
in base.py
which is called from
from .base import LinearRegression
in linear_model/init.py

ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so, 2): Symbol not found: ATL_daxpy
Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so
Expected in: flat namespace
in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so
File "/Users/udi/Documents/MyProjects/Edgar/xbrl2scikit.py", line 33, in
from sklearn import linear_model
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/__init_
.py", line 12, in
from .base import LinearRegression
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/base.py", line 28, in
from cd_fast import sparse_std

@agramfort
scikit-learn member

how did you install scikit-learn?

@udibr

pip install -U scikit-learn
the resulting version is indeed 0.12
this caused the exception (my python is 2.7.3)
I then did
pip uninstall scikit-learn
and downloaded 0.11 tar and installed it using pip and that worked great for me.

@amueller
scikit-learn member

@udibr If you'd be willing / interested in installing 0.12 it would be helpful for us to find out what the problem was.
It could be related to your python path.

@amueller
scikit-learn member

sorry didn't read carefully. This is related to blas, probably a linking problem. It would be good if we could fix this in the upcoming bug fix release 0.12.1 so your feedback would be very welcome.

@ezralanglois

Hi, I have the same issue. I am using ACML for the optimize BLAS/LAPACK library.

It works fine with NumPy and SciPy but not with version 12.

I just ran the following and placed the directory on my PYTHONPATH.

python setup.py build_ext --inplace

Partial import of sklearn during the build process.
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in /usr/local/lib64
libraries mkl,vml,guide not found in /usr/local/lib
libraries mkl,vml,guide not found in /usr/lib64
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
NOT AVAILABLE

atlas_blas_info:
libraries f77blas,cblas,atlas not found in /usr/local/lib64
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib64
libraries f77blas,cblas,atlas not found in /usr/lib
NOT AVAILABLE

/usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:1417: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.doc)
blas_info:
FOUND:
libraries = ['cblas', 'acml_mp', 'acml_mv']
library_dirs = ['/opt/acml4.4.0/gfortran64_mp/lib']
language = f77

FOUND:
libraries = ['cblas', 'acml_mp', 'acml_mv']
library_dirs = ['/opt/acml4.4.0/gfortran64_mp/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77

sklearn/setup.py:64: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
warnings.warn(BlasNotFoundError.doc)

And I get the following error:

import sklearn.manifold

File "/home/robertl/scikit-learn-0.12.1/sklearn/manifold/init.py", line 6, in
from .isomap import Isomap
File "/home/robertl/scikit-learn-0.12.1/sklearn/manifold/isomap.py", line 12, in
from ..decomposition import KernelPCA
File "/home/robertl/scikit-learn-0.12.1/sklearn/decomposition/init.py", line 10, in
from .sparse_pca import SparsePCA, MiniBatchSparsePCA
File "/home/robertl/scikit-learn-0.12.1/sklearn/decomposition/sparse_pca.py", line 8, in
from ..linear_model import ridge_regression
File "/home/robertl/scikit-learn-0.12.1/sklearn/linear_model/init.py", line 12, in
from .base import LinearRegression
File "/home/robertl/scikit-learn-0.12.1/sklearn/linear_model/base.py", line 28, in
from cd_fast import sparse_std
ImportError: /home/robertl/scikit-learn-0.12.1/sklearn/linear_model/cd_fast.so: undefined symbol: dger_

@ezralanglois

I have a workaround for the linking problem (example for bash shell):

export LDFLAGS="-shared -L/opt/acml4.4.0/gfortran64_mp/lib -lacml_mp -lacml_mv -lcblas"

Basically the lapack and blas libraries are not being carried through to the linking stage. In the case above, I am using ACML so the library names are different.

@agramfort
scikit-learn member
@GaelVaroquaux
scikit-learn member
@amueller
scikit-learn member

I don't really understand this. Was numpy linked against the same blas or another? Shouldn't we link against the same blas as numpy?

@mathrho

in file sklearn/linear_model/setup.py
adding atlas lib for compiling solves the problem:
cblas_libs.append('atlas')

@amueller amueller modified the milestone: 0.15.1, 0.14 Jul 18, 2014
@amueller
scikit-learn member

Can anyone comment on whether adding atlas to the libs manually is a good idea?

@ogrisel
scikit-learn member

Can anyone comment on whether adding atlas to the libs manually is a good idea?

I don't understand how adding atlas would fix the build when the user would explicitly want to link against ACML.

Note that the original poster reported an issue to build sklearn against OSX's vecLib / Accelerate using pip install.

Nowadays installing scikit-learn from source build without any issue under OSX and numpy, scipy and scikit-learn provided pre-built whl packages that are automatically used by pip under OSX when doing pip install numpy scipy scikit-learn.

Closing this issue. Adding support for building against ACML would be interesting but should be tackled in a separate issue / pull request.

@ogrisel ogrisel closed this Jul 29, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment