Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

undefined symbol: PyFPE_jbuf #7542

Closed
dsquareindia opened this issue Sep 30, 2016 · 16 comments

Comments

@dsquareindia
Copy link
Contributor

commented Sep 30, 2016

Hi!

I always encounter this problem when I pull from upstream. Following are the version details:

Linux-4.4.0-38-generic-x86_64-with-debian-stretch-sid
('Python', '2.7.12 | packaged by conda-forge | (default, Sep  8 2016, 14:22:31) \n[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]')
('NumPy', '1.11.1')
('SciPy', '0.18.0')
('Scikit-Learn', '0.19.dev0')
('Cython', '0.24.1')

Following is the problem I'm facing:

In [1]: from sklearn import linear_model
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-a6ebbebad697> in <module>()
----> 1 from sklearn import linear_model

/home/devashish/EXPERIMENTATION/scikit-learn/sklearn/linear_model/__init__.py in <module>()
     10 # complete documentation.
     11 
---> 12 from .base import LinearRegression
     13 
     14 from .bayes import BayesianRidge, ARDRegression

/home/devashish/EXPERIMENTATION/scikit-learn/sklearn/linear_model/base.py in <module>()
     30 from ..utils.validation import FLOAT_DTYPES
     31 from ..utils import check_random_state
---> 32 from ..utils.extmath import safe_sparse_dot
     33 from ..utils.sparsefuncs import mean_variance_axis, inplace_column_scale
     34 from ..utils.fixes import sparse_lsqr

/home/devashish/EXPERIMENTATION/scikit-learn/sklearn/utils/extmath.py in <module>()
     24 from ._logistic_sigmoid import _log_logistic_sigmoid
     25 from ..externals.six.moves import xrange
---> 26 from .sparsefuncs_fast import csr_row_norms
     27 from .validation import check_array
     28 from ..exceptions import NonBLASDotWarning

ImportError: sklearn/utils/sparsefuncs_fast.so: undefined symbol: PyFPE_jbuf

I have already executed sudo make and sudo python setup.py develop however I still face this issue. Currently I solve this by cloning the fork and setting up again which is quite cumbersome.

How can this be solved? Sorry if I missed out on stating any detail. Thanks for the help!

@rth

This comment has been minimized.

Copy link
Member

commented Sep 30, 2016

This can apparently happen if you have two versions of numpy installed one with conda and another one with pip.

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Sep 30, 2016

@rth thanks but I'm still getting the same error 😞

@rth

This comment has been minimized.

Copy link
Member

commented Sep 30, 2016

It could still have something to do with conflicts between system python, conda-forge installed packages and those installed with pip. Have you tried building scikit learn in a clean Python virtual environment?

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Sep 30, 2016

I uninstalled numpy and scipy and then reinstalled from conda-forge however I'm still getting this error. numpy.__path__ gives the conda path. When I rm -rf the scikit-learn folder, clone it again from my fork and build it, it runs fine.

@amueller

This comment has been minimized.

Copy link
Member

commented Sep 30, 2016

Can you elaborate what you mean from your fork? Like compared to master? And you use the same method to build? This is unlikely to come from some change in code...

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Sep 30, 2016

Oh sorry, yeah by fork I mean the fork of upstream/master. I use sudo make to build and do a sudo python setup.py develop just to make sure everything is synced up. My numpy and scipy paths are the ~/miniconda2/lib/python2.7.../ paths

@amueller

This comment has been minimized.

Copy link
Member

commented Sep 30, 2016

You need only one of these. make builds and python setup.py develop builds and sets the path. Neither needs sudo.
And I'm not sure what you are saying is working and what you are saying is not working.

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2016

Sorry for not being clear before. Here's what's happening:
When I clone my fork of upstream/master and build and set the path using sudo python setup.py develop (I get this error if I don't sudo: error: [Errno 13] Permission denied: 'scikit_learn.egg-info/requires.txt') everything works fine.
However after some time when I pull from upstream/master into my local clone's master and run sudo make (I get this error if I don't sudo: error: [Errno 13] Permission denied: 'build/temp.linux-x86_64-2.7/libcblas.a' Makefile:24: recipe for target 'clean' failed make: *** [clean] Error 1) I start getting the PyFPE_jbuf error.
I don't know how to get rid of this so, currently, to solve the problem I have to rm -rf the local clone, and do the first step again (of cloning my fork and building).

@rth

This comment has been minimized.

Copy link
Member

commented Oct 1, 2016

Could you post the standard output of your python setup.py develop and/or make commands (linked in an external file) ?

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2016

Following are the outputs for make and develop respectively:
make.txt
setup_develop.txt
Thanks for the help!

@rth

This comment has been minimized.

Copy link
Member

commented Oct 1, 2016

@dsquareindia With a sudo preferably, those just have "permission denied" )

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2016

Oh lol sorry for that! Here are the files again:
make.txt
setup_develop.txt

@rth

This comment has been minimized.

Copy link
Member

commented Oct 1, 2016

Yeah, it is an environment conflict: when you build sklearn all your include paths point to system python /usr/local/lib/python2.7/dist-packages/numpy-1.11.0rc2 and yet you use python from ~/miniconda2/lib/python2.7/.
Maybe your $PATH is misconfigured and one of python, cython or pip points to system python instead of the conda-forge environment. Did you prepend ~/miniconda2/bin/ to your $PATH?

Try cloning the scikit learn again, re-installig numpy/scipy/cython/setuptools making sure they are installed in ~/miniconda2 then rebuild scikit learn. All of it should work without sudo. The fact that you are asked for sudo permissions means that you are installing in the system python env which creates this conflict.

@dsquareindia

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2016

Alright so my cython paths seemed all botched up. I corrected them, re-cloned the scikit-learn fork and ran develop. Now it's working fine but I'll get here again if I encounter this issue after pulling from upstream. Thanks a ton for the help @rth! I'll close this issue for now.

@NataliaDiaz

This comment has been minimized.

Copy link

commented Jun 15, 2017

I am encountering the same error when running on Python 2.7 Ubuntu 14.04, after installing anaconda to update matplotlib:
from sklearn.decomposition import PCA File "/home/gpu_center/.local/lib/python2.7/site-packages/sklearn/decomposition/__init__.py", line 7, in <module> from .nmf import NMF, ProjectedGradientNMF, non_negative_factorization File "/home/gpu_center/.local/lib/python2.7/site-packages/sklearn/decomposition/nmf.py", line 24, in <module> from ..utils.extmath import randomized_svd, safe_sparse_dot, squared_norm File "/home/gpu_center/.local/lib/python2.7/site-packages/sklearn/utils/extmath.py", line 26, in <module> from .sparsefuncs_fast import csr_row_norms ImportError: /home/gpu_center/.local/lib/python2.7/site-packages/sklearn/utils/sparsefuncs_fast.so: undefined symbol: PyFPE_jbuf

@rth

This comment has been minimized.

Copy link
Member

commented Jun 15, 2017

after installing anaconda to update matplotlib

That would break things..

The solution is the same as in #7542 (comment) Either use conda (in which case all your installed packages would be in ~/miniconda/ or pip install --user don't mix the two. Removing either ~/.local/lib/python2.7/site-packages/sklearn or your ~/miniconda folder and reinstalling it cleanly should fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.