Python 3.9 Support

[mjsteinbaugh]

Reporting a bug

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included below a minimal working reproducer (if you are unsure how
  to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

I'm seeing this warning pop up for a clean installation of numba with Python 3.9:

python3 -m pip install numba

Collecting numba
  Using cached numba-0.51.2.tar.gz (2.1 MB)
Processing ./.cache/pip/wheels/40/08/53/26580f3607587bd3fa1a18619841d1dcfedcabf2be52f8e2cd/llvmlite-0.34.0-cp39-cp39-linux_x86_64.whl
Processing ./.cache/pip/wheels/a3/17/dd/f2dba23a35bb6008732772ccfb13d3d0e537fbc6919ce6862b/numpy-1.19.2-cp39-cp39-linux_x86_64.whl
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from numba) (50.3.0)
Building wheels for collected packages: numba
  Building wheel for numba (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-t2pl9xsf/numba/setup.py'"'"'; __file__='"'"'/tmp/pip-install-t2pl9xsf/numba/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-3k5ws828
       cwd: /tmp/pip-install-t2pl9xsf/numba/
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-t2pl9xsf/numba/setup.py", line 354, in <module>
      metadata['ext_modules'] = get_ext_modules()
    File "/tmp/pip-install-t2pl9xsf/numba/setup.py", line 87, in get_ext_modules
      import numpy.distutils.misc_util as np_misc
  ModuleNotFoundError: No module named 'numpy'
  ----------------------------------------
  ERROR: Failed building wheel for numba
  Running setup.py clean for numba
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-t2pl9xsf/numba/setup.py'"'"'; __file__='"'"'/tmp/pip-install-t2pl9xsf/numba/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
       cwd: /tmp/pip-install-t2pl9xsf/numba
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-t2pl9xsf/numba/setup.py", line 354, in <module>
      metadata['ext_modules'] = get_ext_modules()
    File "/tmp/pip-install-t2pl9xsf/numba/setup.py", line 87, in get_ext_modules
      import numpy.distutils.misc_util as np_misc
  ModuleNotFoundError: No module named 'numpy'
  ----------------------------------------
  ERROR: Failed cleaning build dir for numba
Failed to build numba
Installing collected packages: llvmlite, numpy, numba
    Running setup.py install for numba ... done
  DEPRECATION: numba was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Successfully installed llvmlite-0.34.0 numba-0.51.2 numpy-1.19.2

I can see on the README that these versions are currently recommended:

• Python versions: 3.6-3.8
• llvmlite 0.33.*

Is Python 3.9 and llvmlite 0.34.* not supported? Pip is currently warning about wheel build failure, but numba will install.

[mjsteinbaugh]

For reference, the checklist in #6332 describes the numba 0.52 and 0.53 release cycle plans.

[esc]

@mjsteinbaugh thanks for asking about this. The Numba stack, which includes llvmlite currently does not support being executed on Python 3.9. So, I have modified the title of this issue accordingly and re-phrased it as a feature request. We may, if everything goes well, support Python 3.9 with the next patch release before the end of the year.

[esc]

I should also note, that the modifications to the bytecode with Python 3.9 will mean that even if you manage to install/compile Numba and llvmlite it is unlikely that it will work correctly when used.

[mjsteinbaugh]

OK thanks for the update @esc . I figured that was the case, and I'll stick with Python 3.8.6 for the time being.

[esc]

👍

[mwhudson]

FWIW, llvmlite appears to work ok with Python 3.9 (at least, the test suite passes) but the numba test suite seems to hang: https://buildd.debian.org/status/fetch.php?pkg=numba&arch=amd64&ver=0.51.2-1%2Bb1&stamp=1602743150&raw=0

[stuartarchibald]

FWIW, llvmlite appears to work ok with Python 3.9 (at least, the test suite passes) but the numba test suite seems to hang: https://buildd.debian.org/status/fetch.php?pkg=numba&arch=amd64&ver=0.51.2-1%2Bb1&stamp=1602743150&raw=0

@mwhudson thanks for testing. I've started looking at 3.9 for Numba, there's a few changes but nothing huge, I expect a fair bit Numba will still work ok.

[gimperiale]

Is there a rough ETA for this?

[stuartarchibald]

I raised this yesterday at the Numba core developer meeting (minutes). The aim is to ship a 0.52.1, which is functionally identical to 0.52.0 but with Python 3.9 support added, mid-December or before.

[stuartarchibald]

I've started work on this and have discovered that due to a couple of bytecode instruction sequence changes that this is going to be a lot more work than was initially envisaged. The patch for Python 3.9 is probably going to be large and have associated risks (it needs to rewrite swathes of bytecode sequences). It is therefore the view of the core developers that this change will warrant a full new release so as to go through the full release candidate and community testing process. Unless we can figure out a way to avoid this, which is not looking likely at present, Python 3.9 support will most likely land in Numba 0.53 which will be tagged for RC early in 2021.

Sorry to have to bring this news and thank you all for your understanding.

Technical details:

The main issues stem from some bytecode generation changes that arose as a result of:
https://bugs.python.org/issue39320
Essentially, the patch to CPython is pushing work from the interpreter into the compiler, which helps CPython performance. However, this results in an awkward problem for Numba.

An example:

def foo(a, b):
    t = 10
    z = (1, 2, *a, 3, t, *b, 4)
    return z

print(dis.dis(foo))
print(foo((100, np.zeros(4)), (300, 400)))

On Python < 3.9:

 65           0 LOAD_CONST               1 (10)
              2 STORE_FAST               2 (t)

 66           4 LOAD_CONST               6 ((1, 2))
              6 LOAD_FAST                0 (a)
              8 LOAD_CONST               4 (3)
             10 LOAD_FAST                2 (t)
             12 BUILD_TUPLE              2
             14 LOAD_FAST                1 (b)
             16 LOAD_CONST               7 ((4,))
             18 BUILD_TUPLE_UNPACK       5
             20 STORE_FAST               3 (z)

 67          22 LOAD_FAST                3 (z)
             24 RETURN_VALUE
None
(1, 2, 100, array([0., 0., 0., 0.]), 3, 10, 300, 400, 4)

on Python 3.9:

 65           0 LOAD_CONST               1 (10)
              2 STORE_FAST               2 (t)

 66           4 LOAD_CONST               2 (1)
              6 LOAD_CONST               3 (2)
              8 BUILD_LIST               2
             10 LOAD_FAST                0 (a)
             12 LIST_EXTEND              1
             14 LOAD_CONST               4 (3)
             16 LIST_APPEND              1
             18 LOAD_FAST                2 (t)
             20 LIST_APPEND              1
             22 LOAD_FAST                1 (b)
             24 LIST_EXTEND              1
             26 LOAD_CONST               5 (4)
             28 LIST_APPEND              1
             30 LIST_TO_TUPLE
             32 STORE_FAST               3 (z)

 67          34 LOAD_FAST                3 (z)
             36 RETURN_VALUE
None
(1, 2, 100, array([0., 0., 0., 0.]), 3, 10, 300, 400, 4)

On Python <3.9 BUILD_TUPLE and BUILD_TUPLE_UNPACK are used to assemble the tuple, and all the parts of the tuple that are defined as constants are stored as tuples too. For Numba, it's relatively easy to handle this as it's just a question of looking at how many tuples are on the stack and then generating a sequence of binary additions to compute the result.

On Python 3.9, the tuple is built via creating a list, and then extending/appending to it, and then finally using the new op-code LIST_TO_TUPLE to make the list into a tuple. The problem with this is that Numba cannot magically turn a list into a tuple as the tuple type in Numba must have both the size and the types of all elements known at compile time. Also, lists in Numba must be homogeneous in type, so even were it possible to do a list-to-tuple converter, it'd fail unless all the elements of the list were of the same type and the size of the list were known. As a result this new bytecode sequence needs analysing and then rewriting as a compounding of expressions based on tuples such that the type of the resulting tuple can be statically determined. Rewriting bytecode sequences such as the above is inherently complicated and risky and it is this risk which warrants a new release tag with full release candidates such that they can be well tested.

[leriomaggio]

I am happy to report that I just tested the installation of the the 0.53.0rc1 tag from the repo via pip with Python 3.9 and it works perfectly! All test passes.

Replication info:

Environment: Anaconda Python 3.9
OS:

• macOS Big Sur (Intel Architecture)
• Ubuntu 16.04 (with GPU)

Installation: pip install git+https://github.com/numba/numba.git@0.53.0rc1

Test run: python -m numba.tests.test_runtests

(Note: If you have any other suggestion to run numba tests to check the installation, please let me know! :) )

[gmarkall]

I am happy to report that I just tested the installation of the the 0.53.0rc1 tag from the repo via pip with Python 3.9 and it works perfectly! All test passes.

Many thanks for testing and reporting back!

Test run: python -m numba.tests.test_runtests

(Note: If you have any other suggestion to run numba tests to check the installation, please let me know! :) )

I think this would only have tested the tests of the test suite (I would expect to see 10 tests run by test_runtests - do you see more)? If you run:

python -m numba.tests

is everything still good?

[ickc]

$> python -m numba.tests
python: No module named numba.tests.__main__; 'numba.tests' is a package and cannot be directly executed

[stuartarchibald]

Thanks for testing @leriomaggio

@ickc IIRC the Numba test suite can be invoked with:

$ python -m numba.runtests

---

Helping to test the release candidates:

All information about release candidates passing/failing the Numba test suite on various configurations is incredibly valuable, thanks to anyone who is testing.

One major thing that helps with speeding up moving release candidates to final versions is hearing how well they work with your applications. Numba's own test suite is heavily tested on the Numba build farm, and whilst huge (9k+ tests), it cannot cover all the possible things you can feed a compiler as input. This is why community led testing is so important to the Numba project and something for which we (the maintainers) are hugely grateful.

We're hoping to get wheels posted today/tomorrow and will update this announcement when they are live. If you want to test the release candidates on your own projects, please do follow the instructions that will be posted there and comment on the thread reporting success/failure/issues for your project!

Many thanks for your help :)

[leriomaggio]

Hey @gmarkall ,
thanks a lot for your reply!

I am happy to report that I just tested the installation of the the 0.53.0rc1 tag from the repo via pip with Python 3.9 and it works perfectly! All test passes.

Many thanks for testing and reporting back!

My pleasure, glad it was useful!

Test run: python -m numba.tests.test_runtests
(Note: If you have any other suggestion to run numba tests to check the installation, please let me know! :) )

I think this would only have tested the tests of the test suite (I would expect to see 10 tests run by test_runtests - do you see more)?

Yes, exactly! 10 out of 10 on Linux; 9/10 on macOS + 1 skipped (#5 as for No CUDA) - details below

❯ python -m numba.tests.test_runtests --verbose
test_all (__main__.TestCase) ... ok
test_check_slice (__main__.TestCase) ... ok
test_check_slicing_equivalent (__main__.TestCase) ... ok
test_cuda (__main__.TestCase) ... ok
test_cuda_submodules (__main__.TestCase) ... skipped 'NO CUDA'
test_default (__main__.TestCase) ... ok
test_include_exclude_tags (__main__.TestCase) ... ok
test_module (__main__.TestCase) ... ok
test_random (__main__.TestCase) ... ok
test_subpackage (__main__.TestCase) ... ok

----------------------------------------------------------------------
Ran 10 tests in 35.669s

OK (skipped=1)

If you run:

python -m numba.tests

is everything still good?

So I run first:

$ python -m numba.runtests -l

to see how many tests were detected. I got

• 9419 on macOS
• 10521 on Linux (w/ CUDA)

I then ran the test battery on both the machines:

python -m numba.runtests

UPDATES @esc @gmarkall @stuartarchibald

⚠️ Notes: I simplified the reporting here, getting rid of all the very details of the errors to aid readability of the issue. I logged everything, in case you might want to know more :)

Linux (w/ CUDA 10.2):

Ran 10521 tests in 14595.127s

FAILED (failures=15, errors=4, skipped=525, expected failures=13)

**Linux (w/ CUDA 10.2) & numba == 0.52+Python 3.8` for comparison:

----------------------------------------------------------------------
Ran 10351 tests in 13777.715s

FAILED (failures=15, errors=4, skipped=369, expected failures=11)

I can see a pattern here - at a first glance, there shouldn't be any issue introduced with latest rc on Py3.9 :)
In fact (pls see below)

macOS (Python 3.9 and numba==0.53.0rc1:

----------------------------------------------------------------------
Ran 9419 tests in 20361.352s

FAILED (failures=15, errors=1, skipped=510, expected failures=12)

[gmarkall]

$> python -m numba.tests
python: No module named numba.tests.main; 'numba.tests' is a package and cannot be directly executed

@ickc IIRC the Numba test suite can be invoked with:
$ python -m numba.runtests

Apologies for my "thinko" earlier when I incorrectly wrote down the name of the test module. :-/

[leriomaggio]

$> python -m numba.tests
python: No module named numba.tests.main; 'numba.tests' is a package and cannot be directly executed

@ickc IIRC the Numba test suite can be invoked with:
$ python -m numba.runtests

Apologies for my "thinko" earlier when I incorrectly wrote down the name of the test module. :-/

Oh no no worries at all! I did already figured it out by looking at the test scripts ;)

In the meantime, I am finishing writing down in another reply a couple of quite strange behaviours I noted (truly by chance)..

[leriomaggio]

Ok in the meantime, I found this very strange behaviour!

⚠️ PLEASE let me know if you want me to report this in a separate issue! :)

So, I was trying to install numba 0.53.0rc1 as before in a fresh new Conda Env with Python 3.9 on macOS

(CASE A): Environment with numpy (from conda-forge) already installed.

In this case, the installation of numba FAILS

Here is (an excerpt) of the error log:

Running setup.py install for numba ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/valerio/anaconda3/envs/numba_conda/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-h3hljslb/setup.py'"'"'; __file__='"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-h3hljslb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-record-ad75lll7/install-record.txt --single-version-externally-managed --compile --install-headers /Users/valerio/anaconda3/envs/numba_conda/include/python3.9/numba
         cwd: /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-h3hljslb/
    Complete output (894 lines):
    TBB not found
    Using OpenMP from: /Users/valerio/anaconda3/envs/numba_conda
    running install
    running build
    got version from VCS {'version': '0.53.0rc1', 'full': 'a3d29a7f4b354ca99df3ba567f69e7cf84ce9ebd'}
    running build_py
    creating build
    creating build/lib.macosx-10.9-x86_64-3.9
    creating build/lib.macosx-10.9-x86_64-3.9/numba
    copying numba/_version.py -> build/lib.macosx-10.9-x86_64-3.9/numba
    copying numba/__init__.py -> build/lib.macosx-10.9-x86_64-3.9/numba
    copying numba/__main__.py -> build/lib.macosx-10.9-x86_64-3.9/numba
    copying numba/runtests.py -> build/lib.macosx-10.9-x86_64-3.9/numba
    copying numba/extending.py -> build/lib.macosx-10.9-x86_64-3.9/numba
    creating build/lib.macosx-10.9-x86_64-3.9/numba/misc

OMISSIS

building 'numba._dynfunc' extension
    Warning: Can't read registry to find the necessary compiler setting
    Make sure that Python modules winreg, win32api or win32con are installed.
    C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/valerio/anaconda3/envs/numba_conda/include -fPIC -O2 -isystem /Users/valerio/anaconda3/envs/numba_conda/include

    creating build/temp.macosx-10.9-x86_64-3.9
    creating build/temp.macosx-10.9-x86_64-3.9/numba
    compile options: '-I/Users/valerio/anaconda3/envs/numba_conda/include/python3.9 -c'
    clang: numba/_dynfuncmod.c
    clang -bundle -undefined dynamic_lookup -Wl,-rpath,/Users/valerio/anaconda3/envs/numba_conda/lib -L/Users/valerio/anaconda3/envs/numba_conda/lib -Wl,-rpath,/Users/valerio/anaconda3/envs/numba_conda/lib -L/Users/valerio/anaconda3/envs/numba_conda/lib -arch x86_64 build/temp.macosx-10.9-x86_64-3.9/numba/_dynfuncmod.o -o build/lib.macosx-10.9-x86_64-3.9/numba/_dynfunc.cpython-39-darwin.so
    building 'numba._dispatcher' extension

OMISSIS

error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/valerio/anaconda3/envs/numba_conda/include -fPIC -O2 -isystem /Users/valerio/anaconda3/envs/numba_conda/include -I/Users/valerio/anaconda3/envs/numba_conda/include/python3.9 -c numba/np/ufunc/omppool.cpp -o build/temp.macosx-10.9-x86_64-3.9/numba/np/ufunc/omppool.o -fopenmp -std=c++11" failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/valerio/anaconda3/envs/numba_conda/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-h3hljslb/setup.py'"'"'; __file__='"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-h3hljslb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-record-ad75lll7/install-record.txt --single-version-externally-managed --compile --install-headers /Users/valerio/anaconda3/envs/numba_conda/include/python3.9/numba Check the logs for full command output.

Symptomps (or things I noted)

Don't know if this is related or not, but I noticed a minor release mismatch between PyPi and Conda-forge repos:

• latest numpy installed from PyPi: 1.20.1
• lastet numpy installed from Conda (for Python 3.9): 1.20.0

Besides, installing from conda-forge also installs:

libblas            conda-forge/osx-64::libblas-3.9.0-8_openblas
libcblas           conda-forge/osx-64::libcblas-3.9.0-8_openblas
libgfortran        conda-forge/osx-64::libgfortran-5.0.0-9_3_0_h6c81a4c_17
libgfortran5       conda-forge/osx-64::libgfortran5-9.3.0-h6c81a4c_17
liblapack          conda-forge/osx-64::liblapack-3.9.0-8_openblas
libopenblas        conda-forge/osx-64::libopenblas-0.3.12-openmp_h54245bb_1
llvm-openmp        conda-forge/osx-64::llvm-openmp-11.0.1-h7c73e74_0
numpy              conda-forge/osx-64::numpy-1.20.0-py39h3c955ea_0

which means, llvm dependency is also satisfied. That may be the issue perhaps?

(CASE B): Environment with NO pre-existing numpy installation

In this case (not sure why I did not notice this before - but I repeated this three times and could replicate the same issue):

When I run:

$ pip install git+https://github.com/numba/numba.git@0.53.0rc1

it correctly downloads the dependency - but I got this error while running numba setup.py script.

(full output of the command report below)

Collecting git+https://github.com/numba/numba.git@0.53.0rc1
  Cloning https://github.com/numba/numba.git (to revision 0.53.0rc1) to /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo
  Running command git clone -q https://github.com/numba/numba.git /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo
  Running command git checkout -q 4e5e12553ed30985db2dbb46231109e859bd75b2
Collecting llvmlite<0.37,>=0.36.0rc1
  Using cached llvmlite-0.36.0rc1-cp39-cp39-macosx_10_9_x86_64.whl (18.5 MB)
Collecting numpy>=1.15
  Using cached numpy-1.20.1-cp39-cp39-macosx_10_9_x86_64.whl (16.1 MB)
Requirement already satisfied: setuptools in /Users/valerio/anaconda3/envs/numba/lib/python3.9/site-packages (from numba==0.53.0rc1) (52.0.0.post20210125)
Building wheels for collected packages: numba
  Building wheel for numba (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/valerio/anaconda3/envs/numba/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py'"'"'; __file__='"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-wheel-3h4aoi44
       cwd: /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py", line 420, in <module>
      metadata['ext_modules'] = get_ext_modules()
    File "/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py", line 148, in get_ext_modules
      import numpy.distutils.misc_util as np_misc
  ModuleNotFoundError: No module named 'numpy'
  ----------------------------------------
  ERROR: Failed building wheel for numba
  Running setup.py clean for numba
  ERROR: Command errored out with exit status 1:
   command: /Users/valerio/anaconda3/envs/numba/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py'"'"'; __file__='"'"'/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
       cwd: /private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py", line 420, in <module>
      metadata['ext_modules'] = get_ext_modules()
    File "/private/var/folders/16/xdrp9d8s5510rt6fwr4j6llr0000gn/T/pip-req-build-7edsv4wo/setup.py", line 148, in get_ext_modules
      import numpy.distutils.misc_util as np_misc
  ModuleNotFoundError: No module named 'numpy'
  ----------------------------------------
  ERROR: Failed cleaning build dir for numba
Failed to build numba
Installing collected packages: numpy, llvmlite, numba
    Running setup.py install for numba ... done
  DEPRECATION: numba was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Successfully installed llvmlite-0.36.0rc1 numba-0.53.0rc1 numpy-1.20.1

So numpy is not initially found by the installation script of numba but ultimately all is installed with no error. In fact, if I re-run numba installation all works perfectly.

Symptomps (or, my personal impression on the matter - which means I can be completely wrong!).

I believe that this issue may be somewhat related to PEP 517 and the fact that setuptools dependency specifications are just runtime dependencies rather than build dependencies!

It indeed reminded me something I already experienced some months ago when I decided to slide down the rabbit hole of Python Package manager! 😄
I was working on this toy project of mine for a very simple IPython extension // Jupyter notebook theme, and I was trying to have jupyter as a build dependency (at that time I was also using jupyter.serverextension).
Hence, I resorted using pyproject.toml ==> here is the example I was referring to.

Hope this helps!

[ickc]

Do you want us to report test results and what info should we be pasting?

e.g. I ran on a few different platforms and 2 of them has the following results:

----------------------------------------------------------------------
Ran 9373 tests in 16430.795s

FAILED (failures=15, skipped=487, expected failures=12)

----------------------------------------------------------------------
Ran 9445 tests in 24394.475s

FAILED (failures=15, errors=2, skipped=429, expected failures=12)

Edit:

Here is 2 example logs. These output should be similar to what can be obtained by (although I actually capture them after the fact so I only have the last ~2000 lines.)

{
    mamba create -c conda-forge -c numba -n numba-0-53 python=3.9 'numba=0.53.0rc1' cudatoolkit -y
    conda activate numba-0-53
    command time -v python -m numba.runtests
    neofetch | sed -e "s/$USER/numba/g" -e "s/$HOSTNAME/numba/g"
    conda list
} |& sed "s/\/[^\" ]*$USER/~/g" | tee numba-test-report.log

numba-test-report-3650.log
numba-test-report-E5-2698v3.log
numba-test-report-thinkpad.log

summaries:

• more failures than expected
• many of them is related to memory leaks. Should we be concerning here to use it in production?
• many others are assertion related to comparing inf with nan, probably safe to ignore?

[leriomaggio]

@ickc I basically got the same, but also with numba 0.52 and Py3.8 so there is nothing apparently introduced in this release.

[stuartarchibald]

Could everyone please direct RC testing information based on this post (this trying to keep all the information together :-) ): https://numba.discourse.group/t/helping-test-numba-0-53-0-rc/519

Summary:

• Please open specific issues related to the RCs as new issues.
• Please report successes for your projects on the discourse thread.
• Please open a new issue for anything that's a bug/problem and not RC testing specific.

For general help, usual channels:

• https://numba.discourse.group for more involved things.
• https://gitter.im/numba/numba for quick things/needing interactive help.

Many thanks for your cooperation and help in testing Numba!

[jondo]

At lmcinnes/pynndescent#115 (comment), @milianw asks whether it was considered to release a 0.51.3 that warns about this incompatibility. (It would have also helped me.)

[esc]

@jondo you mean, an additional runtime check in case the package does somehow get past pip? (not unheard of: #6708) I don't think that was considered. It might be a good idea, but it might also be too late now for 0.51.3? But perhaps a runtime check to exclude unsupported Python versions (e.g. Python 3.10) might be a good idea for the future.

[milianw]

@esc yes that would be great. I never saw any trouble during updating/installing numba via pip, despite the fact that I have python 3.9.1 installed:

$ python --version
Python 3.9.1
$ pip install --user numba
Requirement already satisfied: numba in /home/milian/.local/lib/python3.9/site-packages (0.51.2)
...

I just uninstalled numba and reinstalled it via pip - no complaint there either. Yet it clearly doesn't work as indicated by this bug report here.

[AlexanderUp]

Good day, guys!

I ran numba tests and faced some errors, please see file attached for details (file contains console output).

All tests were run on macos Catalina 10.15.7 (iMac 27 2017 Intel CPU) with Python 3.9.2 version and Numba 0.53.0rc1.

Short results:

python3 -m numba.tests.test_runtests --verbose
test_all (__main__.TestCase) ... ok
test_check_slice (__main__.TestCase) ... ok
test_check_slicing_equivalent (__main__.TestCase) ... ok
test_cuda (__main__.TestCase) ... ok
test_cuda_submodules (__main__.TestCase) ... skipped 'NO CUDA'
test_default (__main__.TestCase) ... ok
test_include_exclude_tags (__main__.TestCase) ... ok
test_module (__main__.TestCase) ... ok
test_random (__main__.TestCase) ... ok
test_subpackage (__main__.TestCase) ... ok

----------------------------------------------------------------------
Ran 10 tests in 38.823s

OK (skipped=1)

python3 -m numba.runtests -l

9419 tests found. 9419 selected

Ran 9419 tests in 1537.611s

FAILED (failures=10, skipped=479, expected failures=12)

numba_tests.txt

[stuartarchibald]

@AlexanderUp thanks for testing, any chance you could please report the results (along with Python version, NumPy version and where you got Numba from) on this discussion thread https://numba.discourse.group/t/helping-test-numba-0-53-0-rc/519/18 that's tracking RC progress. Many thanks.

[hjaffar]

It's seems I have the same issue , is there any way to overcome this issue ? below the error message

python version: Python 3.10.4
and this list of dependencies that I am using :
asgiref==3.5.0
autopep8==1.6.0
cx-Oracle==8.3.0
Django==4.0.2
django-daterange-filter==1.3.0

Error message:
-e git+https://github.com/django-debug-toolbar/django-debug-toolbar.git@bf785130d5c60e4c269be154d231775d02008801#egg=django_debug_toolbar
django-debug-toolbar-force==0.1.8
django-nine==0.2.5
psycopg2==2.9.3
pycodestyle==2.8.0
six==1.16.0
sqlparse==0.4.2
toml==0.10.2
tzdata==2021.5

Exception in thread django-main-thread:
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1264.0_x64__qbz5n2kfra8p0\lib\threading.py", line 1009, in _bootstrap_inner
self.run()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1264.0_x64__qbz5n2kfra8p0\lib\threading.py", line 946, in run
self._target(*self._args, **self.kwargs)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\utils\autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\core\management\commands\runserver.py", line 124, in inner_run
self.check(display_num_errors=True)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\core\management\base.py", line 438, in check
all_issues = checks.run_checks(
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\core\checks\registry.py", line 77, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\core\checks\urls.py", line 13, in check_url_config
return check_resolver(resolver)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\core\checks\urls.py", line 23, in check_resolver
return check_method()
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\urls\resolvers.py", line 446, in check
for pattern in self.url_patterns:
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\utils\functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\urls\resolvers.py", line 632, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\utils\functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\urls\resolvers.py", line 625, in urlconf_module
return import_module(self.urlconf_name)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1264.0_x64__qbz5n2kfra8p0\lib\importlib_init.py",
line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "C:\Users\ASUS\Desktop\isehasa_project\isehasa\urls.py", line 10, in
path('configuration/',include('configurations.urls')),
File "C:\Users\ASUS\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\django\urls\conf.py", line 34, in include
urlconf_module = import_module(urlconf_module)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1264.0_x64__qbz5n2kfra8p0\lib\importlib_init.py",
line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "C:\Users\ASUS\Desktop\isehasa_project\configurations\urls.py", line 2, in
from . import views
File "C:\Users\ASUS\Desktop\isehasa_project\configurations\views.py", line 8, in
from configurations.add_resources import add_resource,resource
File "C:\Users\ASUS\Desktop\isehasa_project\configurations\add_resources.py", line 9, in
from numba import jit
ModuleNotFoundError: No module named 'numba'

[sklam]

@hjaffar, it seems numba may not have been installed to your python environment. Did you install python through pip? Were there any error when installing numba?

[hjaffar]

yes I installed numba , my issue solve after stop and start the server again