Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken package: numpy #9

Open
liffiton opened this issue Jan 12, 2019 · 12 comments

Comments

Projects
None yet
5 participants
@liffiton
Copy link

commented Jan 12, 2019

Broken package
Package name: numpy
Version: 1.15.4
More information:

I get an ImportError when importing numpy. I have chased down whatever non-python package dependencies I could, and the error doesn't look like a missing library. I have libatlas-base-dev installed. The error:

Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
ImportError: /usr/local/lib/python3.5/dist-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so: undefined symbol: cblas_sgemm

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python3.5/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python3.5/dist-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python3.5/dist-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: /usr/local/lib/python3.5/dist-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so: undefined symbol: cblas_sgem

In case its helpful:

$ ldd /home/pi/.local/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so
        linux-vdso.so.1 (0x7ebf2000)
        /usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76e0a000)
        libf77blas.so.3 => /usr/lib/libf77blas.so.3 (0x76de1000)
        libcblas.so.3 => /usr/lib/arm-linux-gnueabihf/libcblas.so.3 (0x76d87000)
        libatlas.so.3 => /usr/lib/libatlas.so.3 (0x76b0f000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76a90000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76a67000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76928000)
        /lib/ld-linux-armhf.so.3 (0x76f93000)
        libgfortran.so.3 => /usr/lib/arm-linux-gnueabihf/libgfortran.so.3 (0x76873000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76846000)
        libf2c.so.2 => /usr/lib/arm-linux-gnueabihf/libf2c.so.2 (0x7681d000)

This is on Raspbian Lite.

@bennuttall

This comment has been minimized.

Copy link
Collaborator

commented Jan 12, 2019

It looks like you have two instances of numpy installed. Your post includes two different paths for numpy:

/usr/local/lib/python3.5/dist-packages/numpy
/home/pi/.local/lib/python3.5/site-packages/numpy

Did you install with sudo and without? I would purge one and try again, and failing that, purge the other and reinstall.

@liffiton

This comment has been minimized.

Copy link
Author

commented Jan 12, 2019

Ah, sorry for the red herring. The two shell traces were from different installations of the same library in different places.

To be certain, I've purged every installed version (verified in the first command below) and re-installed:

$ python3 -m numpy
/usr/bin/python3: No module named numpy

$ pip3 install --user numpy
Collecting numpy
  Using cached https://www.piwheels.org/simple/numpy/numpy-1.15.4-cp35-cp35m-linux_armv7l.whl
Installing collected packages: numpy
Successfully installed numpy-1.15.4

$ python3 -m numpy
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
ImportError: /home/pi/.local/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so: undefined symbol: cblas_sgemm

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.5/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.5/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/home/pi/.local/lib/python3.5/site-packages/numpy/core/__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: /home/pi/.local/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so: undefined symbol: cblas_sgemm

$ ldd /home/pi/.local/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-arm-linux-gnueabihf.so
        linux-vdso.so.1 (0x7eda3000)
        /usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76d58000)
        libf77blas.so.3 => /usr/lib/libf77blas.so.3 (0x76d2f000)
        libcblas.so.3 => /usr/lib/arm-linux-gnueabihf/libcblas.so.3 (0x76cd5000)
        libatlas.so.3 => /usr/lib/libatlas.so.3 (0x76a5d000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x769de000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x769b5000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76876000)
        /lib/ld-linux-armhf.so.3 (0x76ee1000)
        libgfortran.so.3 => /usr/lib/arm-linux-gnueabihf/libgfortran.so.3 (0x767c1000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76794000)
        libf2c.so.2 => /usr/lib/arm-linux-gnueabihf/libf2c.so.2 (0x7676b000)

@bennuttall bennuttall changed the title Broken package numpy Broken package: numpy Jan 12, 2019

@bennuttall

This comment has been minimized.

Copy link
Collaborator

commented Jan 13, 2019

I just tested it and worked fine for me.

Is there any chance you could have not deleted all of numpy, perhaps left the multiarray bit behind? Try deleting everything in /home/pi/.local/lib/python3.5/site-packages/numpy.

You could try installing in a virtualenv:

sudo apt install python3-virtualenv -y
virtualenv -p python3 numpy-env
source numpy-env/bin/activate
pip install numpy

That way no other path can be used.

If you think piwheels is at fault you should try building it yourself:

pip3 install numpy --upgrade --no-binary :all:

It takes about 20 mins on a Pi 3.

@liffiton

This comment has been minimized.

Copy link
Author

commented Jan 13, 2019

Building it manually seems to work.

First I verified there was no multiarray anything sticking around:

$ find -name "*multiarray*"    # in my home directory
$ find /usr -name "*multiarray*"

Both found no results.

Then after building manually, importing numpy worked fine:

$ pip3 install numpy --user --no-binary :all:
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/2d/80/1809de155bad674b494248bcfca0e49eb4c5d8bee58f26fe7a0dd45029e2/numpy-1.15.4.zip (4.5MB)
    100% |████████████████████████████████| 4.5MB 76kB/s
Skipping bdist_wheel for numpy, due to binaries being disabled for it.
Installing collected packages: numpy
  Running setup.py install for numpy ... done
Successfully installed numpy-1.15.4

$ python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>>
@bennuttall

This comment has been minimized.

Copy link
Collaborator

commented Jan 13, 2019

Weird. Anything unusual about your usage we should know?

Does uninstalling and re-installing from piwheels still error as before?

@liffiton

This comment has been minimized.

Copy link
Author

commented Jan 14, 2019

Same issue (with cblas_sgemm) after uninstall and install from piwheels.

I installed Raspbian Lite, latest version, and the only relevant thing I can think of is that I've installed and uninstalled and reinstalled several blas/lapack/etc libraries while trying to get opencv working also. One thought I've had is that there might be an issue with the different blas libs available? Currently, I have:

$ apt list *blas* | grep installed
libblas-common/stable,now 3.7.0-2 armhf [installed,automatic]
libblas3/stable,now 3.7.0-2 armhf [installed,auto-removable]
libcblas-dev/stable,now 3.2.1+dfsg-1 armhf [installed]
libcblas3/stable,now 3.2.1+dfsg-1 armhf [installed,automatic]

libopenblas is also available, and possibly others? I'm not entirely clear on what's what with those.

@RobertLucian

This comment has been minimized.

Copy link

commented Jun 25, 2019

Coming in here, I also have the same issue as @liffiton. No matter how many times I retry reinstalling the numpy package (the wheel) and its dependencies, it never works. As of this moment, the latest version is 1.16.4.

I have tried a bunch of different versions with the hope that it may be something going bad with the recent versions of it. As I have suspected, a version that still works is 1.13.3. I also tried version 1.13.1 and it functions and I reckon all previous versions work too.

So I further suspect there's something bad going in the building process of numpy, either coming from the library itself or from a dependency of it.

I thought I might drop this observation here so that others know it ain't their fault when it fails to import the package.

@tamasgal

This comment has been minimized.

Copy link

commented Jun 27, 2019

I have the same problem... trying to build numpy from source to find the bug for a memory leak in the recarray structure and I am not able to import it. The compilation is working without errors. This is what I get when I try to import numpy (I also removed it a few times and tried with virtualenvs etc.):

~/Dev/numpy/numpy/core/__init__.py in <module>
     45 """ % (sys.version_info[0], sys.version_info[1], sys.executable,
     46         __version__, exc)
---> 47     raise ImportError(msg)
     48 finally:
     49     for envkey in env_added:

ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
  1. Check that you expected to use Python3.7 from "/home/tgal/.pyenv/versions/3.7.3/bin/python3.7",
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy version "1.17.0.dev0+4178b5a" you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.

Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.

Original error was: /home/tgal/Dev/numpy/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm
@RobertLucian

This comment has been minimized.

Copy link

commented Jun 27, 2019

@tamasgal out of curiosity, do you really need to have a specific version of numpy? Cause if you don't, you can use an older version like I did and get away with it. This until the package gets fixed of course.

@tamasgal

This comment has been minimized.

Copy link

commented Jun 27, 2019

I can live with that of course, my comment was more like a reference for future... ;)

@Florina1996

This comment has been minimized.

Copy link

commented Jul 5, 2019

@RobertLucian can you recomend a stable version of numpy?
I do have the same error as @tamasgal

@RobertLucian

This comment has been minimized.

Copy link

commented Jul 5, 2019

@Florina1996 I can't say with accuracy which one is stable, but regarding the cblas_sgemm error, the 1.13.3 version works fine for me.

@bennuttall bennuttall transferred this issue from piwheels/piwheels Jul 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.