numba==0.55.0 is broken with numpy==1.19.5 on Python 3.7

[ascillitoe]

The latest numba version (0.55.0) is broken when used with numpy==1.19.5 on Python 3.7 (e.g. a colab enviroment!).

Steps to reproduce

On a x86_64 linux machine:

1. virtualenv -p 3.7 venv
2. source venv/bin/activate
3. pip install numpy==1.19.5 numba==0.55.0
4. In the python shell: import numba

Error

RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/__init__.py", line 38, in <module>
    from numba.core.decorators import (cfunc, generated_jit, jit, njit, stencil,
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/core/decorators.py", line 12, in <module>
    from numba.stencils.stencil import stencil
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/stencils/stencil.py", line 11, in <module>
    from numba.core import types, typing, utils, ir, config, ir_utils, registry
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/core/ir_utils.py", line 16, in <module>
    from numba.core.extending import _Intrinsic
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/core/extending.py", line 19, in <module>
    from numba.core.pythonapi import box, unbox, reflect, NativeValue  # noqa: F401
  File "/home/ascillitoe/Documents/Work/alibi_detect_test/venv3.7/lib/python3.7/site-packages/numba/core/pythonapi.py", line 12, in <module>
    from numba import _helperlib
ImportError: numpy.core.multiarray failed to import

More details

• The above works fine on Python 3.8 and 3.9.
• The specifc wheel pip used was numba-0.55.0-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.
• Running !pip install numba==0.55.0 and then import numba in Google colab yields the same error (numpy==1.19.5 is the default here).

[esc]

@ascillitoe thank you for reporting this. I tried to replicate this on my local linux machine, and I received:

💥 zsh» python3.7 -m venv venv
esc@sunburst /tmp 
💥 zsh» source venv/bin/activate
esc@sunburst /tmp 
💥 zsh» which pip
/tmp/venv/bin/pip
esc@sunburst /tmp 
💥 zsh» pip install numpy==1.19.5 numba==0.55.0
Collecting numpy==1.19.5
  Downloading https://files.pythonhosted.org/packages/b1/e1/8c4c5632adaffc18dba4e03e97458dc1cb00583811e6982fc620b9d88515/numpy-1.19.5-cp37-cp37m-manylinux1_x86_64.whl (13.4MB)
    100% |████████████████████████████████| 13.4MB 111kB/s 
Collecting numba==0.55.0
  Downloading https://files.pythonhosted.org/packages/cf/e2/8213c08fc9392c99b37de9823119a83576469354154eb08ad653b6ab5213/numba-0.55.0.tar.gz (2.3MB)
    100% |████████████████████████████████| 2.3MB 584kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-kb36p3gg/numpy-1.22.0/setup.py", line 34, in <module>
    
    RuntimeError: Python version >= 3.8 required.
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-v_58caot/numba/setup.py", line 415, in <module>
        setup(**metadata)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 128, in setup
        _install_setup_requires(attrs)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/dist.py", line 513, in fetch_build_eggs
        replace_conflicting=True,
      File "/tmp/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 774, in resolve
        replace_conflicting=replace_conflicting
      File "/tmp/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1057, in best_match
        return self.obtain(req, installer)
      File "/tmp/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1069, in obtain
        return installer(requirement)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/dist.py", line 580, in fetch_build_egg
        return cmd.easy_install(req)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 698, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 724, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 909, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1177, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1163, in run_setup
        run_setup(setup_script, args)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/usr/lib/python3.7/contextlib.py", line 130, in __exit__
        self.gen.throw(type, value, traceback)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/lib/python3.7/contextlib.py", line 130, in __exit__
        self.gen.throw(type, value, traceback)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/_vendor/six.py", line 685, in reraise
        raise value.with_traceback(tb)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/tmp/venv/lib/python3.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-kb36p3gg/numpy-1.22.0/setup.py", line 34, in <module>
    
    RuntimeError: Python version >= 3.8 required.
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-v_58caot/numba/
pip install numpy==1.19.5 numba==0.55.0  5,40s user 0,67s system 58% cpu 10,288 total

[esc]

@ascillitoe thank you again, I can reproduce this on:

quay.io/pypa/manylinux2010_x86_64 latest d6e8d4174693 17 months ago 1.01GB

The RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd looks suspicious.

(venv) [root@1d80fed031ac ~]# pip install numpy==1.19.5 numba==0.55.0
Collecting numpy==1.19.5
  Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8 MB)
     |████████████████████████████████| 14.8 MB 8.4 MB/s
Collecting numba==0.55.0
  Downloading numba-0.55.0-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.3 MB)
     |████████████████████████████████| 3.3 MB 11.1 MB/s
Requirement already satisfied: setuptools in ./venv/lib/python3.7/site-packages (from numba==0.55.0) (47.1.0)
Collecting llvmlite<0.39,>=0.38.0rc1
  Downloading llvmlite-0.38.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
     |████████████████████████████████| 34.5 MB 7.3 MB/s
Installing collected packages: numpy, llvmlite, numba
Successfully installed llvmlite-0.38.0 numba-0.55.0 numpy-1.19.5
WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/root/venv/bin/python3.7 -m pip install --upgrade pip' command.
(venv) [root@1d80fed031ac ~]# python
Python 3.7.12 (default, Nov 15 2021, 14:48:31)
[GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numba
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/venv/lib/python3.7/site-packages/numba/__init__.py", line 38, in <module>
    from numba.core.decorators import (cfunc, generated_jit, jit, njit, stencil,
  File "/root/venv/lib/python3.7/site-packages/numba/core/decorators.py", line 12, in <module>
    from numba.stencils.stencil import stencil
  File "/root/venv/lib/python3.7/site-packages/numba/stencils/stencil.py", line 11, in <module>
    from numba.core import types, typing, utils, ir, config, ir_utils, registry
  File "/root/venv/lib/python3.7/site-packages/numba/core/ir_utils.py", line 16, in <module>
    from numba.core.extending import _Intrinsic
  File "/root/venv/lib/python3.7/site-packages/numba/core/extending.py", line 19, in <module>
    from numba.core.pythonapi import box, unbox, reflect, NativeValue  # noqa: F401
  File "/root/venv/lib/python3.7/site-packages/numba/core/pythonapi.py", line 12, in <module>
    from numba import _helperlib
ImportError: numpy.core.multiarray failed to import

[seibert]

@esc: We need to build the numba wheel with an older numpy in order to fix this, I think.

[ascillitoe]

Thanks for the quick response @esc and @seibert!

[seibert]

Although, looking more closely at our wheel building automation, we should be building the Python 3.7 wheel with numpy 1.17, so somehow that isn't happening?

[seibert]

OK, I think I see what is happening on our internal system. Will confer with @esc and see if we can get a new wheel posted next week.

[esc]

@ascillitoe the fix was easy and I have uploaded a new build for that combination of Python and platform. It works locally in my docker:

(venv) [root@64d990b23f2e ~]# pip install numpy==1.19.5 numba==0.55.0
Collecting numpy==1.19.5
  Downloading numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8 MB)
     |████████████████████████████████| 14.8 MB 4.8 MB/s
Collecting numba==0.55.0
  Downloading numba-0.55.0-1-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.3 MB)
     |████████████████████████████████| 3.3 MB 8.1 MB/s
Collecting llvmlite<0.39,>=0.38.0rc1
  Downloading llvmlite-0.38.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
     |████████████████████████████████| 34.5 MB 7.6 MB/s
Requirement already satisfied: setuptools in ./venv/lib/python3.7/site-packages (from numba==0.55.0) (47.1.0)
Installing collected packages: numpy, llvmlite, numba
Successfully installed llvmlite-0.38.0 numba-0.55.0 numpy-1.19.5
WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/root/venv/bin/python3.7 -m pip install --upgrade pip' command.
(venv) [root@64d990b23f2e ~]# python
Python 3.7.12 (default, Nov 15 2021, 14:48:31)
[GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import numba

As you can see the build 1 was picked correctly by pip in this case:

  Downloading numba-0.55.0-1-cp37-cp37m-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.3 MB)

Can you give it a try?

[ascillitoe]

Hi @esc, import numba works for me in a 3.7 virtual env on my local machine now, and the relevant tests in our CI also appear to be passing too. Many thanks for fixing this so quickly!