Need meta-data patches for Numba/NumPy version compatibility?

[stuartarchibald]

As discussed RE TBB recently (#87 (comment)), seems like the conda-forge Numba packages might similarly need a meta-data fix to ensure that compatible NumPy builds are installed. xref bug report: numba/numba#7830

Reproducer for the invalid environment:

conda create -n _tmp -c conda-forge numba=0.53.1 python=3.8

this resolves to using numpy-1.22.2 locally.

I'm not sure how far this issue extends with respect to Numba versions at present.

CC @jakirkham @henryiii

[henryiii]

Currently we have this locally:

numba-feedstock/recipe/meta.yaml

Lines 63 to 64 in a788925

	# numpy has a hard upper limit currently
	- numpy >=1.18,<1.22

But it wasn't there for easier versions. Is there a place to see all of the version compat info?

[jakirkham]

In the issue they are referencing the numba version 0.53.1 package

The constraint appears to have been added in 0.54.0 ( #83 ). So after the version the user got

The answer is probably to hot-fix older numba packages to make sure they have some numpy constraint (though we might need help in building out that list or doing a rougher patch that would be good enough for practical purposes). Matt mentioned this with a few particular versions in issue ( conda-forge/conda-forge-repodata-patches-feedstock#207 )

We had explored hot-fixing older numba packages in PR ( conda-forge/conda-forge-repodata-patches-feedstock#51 ), but we didn't go through with it. Sounds like we may want a change like that after all

[stuartarchibald]

But it wasn't there for easier versions. Is there a place to see all of the version compat info?

This is a good question and the answer was unfortunately no! I think this is a really useful thing to have and maintain as part of the release process for Numba so I've tried creating such a table based on historical metadata.yamls and change logs. PR for it is here: numba/numba#7833

The current attempt at assembling the version information can be seen rendered in:
https://github.com/numba/numba/blob/d8220616097e5737f760dda1f378857614c26426/docs/source/reference/support_info.rst

[beckermr]

I just hit this bug with numba 0.53.* and numpy 1.22.*

[stuartarchibald]

The PR with the docs numba/numba#7833 has been merged now so can act as a reference for constraints on the meta-data.

[jakirkham]

So the next thing to do would be to add this as a hot-fix to the repodata.

Given the oldest version covered in that table is 0.47, are we comfortable applying that NumPy exclusive upper bound of 1.18 to Numba versions before 0.47? Asking since part of the issue here is we need an upper bound for all Numba versions (even older ones).

[stuartarchibald]

So the next thing to do would be to add this as a hot-fix to the repodata.

Given the oldest version covered in that table is 0.47, are we comfortable applying that NumPy exclusive upper bound of 1.18 to Numba versions before 0.47? Asking since part of the issue here is we need an upper bound for all Numba versions (even older ones).

hmmm I guess this is going to just require a pragmatic decision given information prior to 0.47 would require a degree of archeology :) As Numba has a reliance on NumPy and 0.47 is from two years ago, I think it would be reasonable to use that a base version for the metadata as it fits roughly with NumPy's NEP-0029. It's also the last version that supports Python 2.7 so covers that use case (though I think the percentage of Python 2.7 downloads is ~<1%). In summary, I'm of the view we should go with NumPy 1.18 unless anyone has objection/proposes something else to discuss?

Also, as this is meta-data patching, I assume if there's some issue it can be patched again, albeit at the cost of having to do that!

Thanks for your help @jakirkham !

[beckermr]

Yes we can change the patches at any time.

[hmaarrfk]

xref to tracking: numba/numba#7754

[ThomasGandilhon]

Hi,
I am encoutering the issue aswell with python 3.8, numba 0.54.1 and numpy 1.22. For some reasons conda will allow this env to be created, but numba will fail to run.

After reading about the issue, I am struggling to identify the next steps to fix this ? Should a PR be raised on the repodata patches, along the line of the one previously mentionned ? (conda-forge/conda-forge-repodata-patches-feedstock#51)

I am willing to raise the PR, just not sure where this topic is standing at right now.

[tomvothecoder]

Hello, just giving my input on this issue.

As a workaround, I had to constrain numba >=0.55.2 for my package's conda-forge recipe in order to support numpy >=1.22, This is similar to what others have mentioned.

Here's the related issue and PR:

• “ImportError: Numba needs NumPy 1.21 or less” xcdat-feedstock#5
• Constrain numba >=0.55.2 for numpy >=1.22 compatibility xcdat-feedstock#9

Here's a snippet of the numba version compatibility table for reference:

Source: https://numba.readthedocs.io/en/stable/user/installing.html#version-support-information

Numba	Release date	Python	NumPy	llvmlite	LLVM	TBB
0.57.x	TBC	3.8.x <= version < 3.12	1.19 <= version < 1.24	0.40.x	11.x	2021.x
0.56.x	2022-07-25	3.7.x <= version < 3.11	1.18 <= version < 1.23	0.39.x	11.x	2021.x
0.55.2	2022-05-25	3.7.x <= version < 3.11	1.18 <= version < 1.23	0.38.x	11.x	2021.x
0.55.{0,1}	2022-01-13	3.7.x <= version < 3.11	1.18 <= version < 1.22	0.38.x	11.x	2021.x
0.54.x	2021-08-19	3.6.x <= version < 3.10	1.17 <= version < 1.21	0.37.x	11.x	2021.x
0.53.x	2021-03-11	3.6.x <= version < 3.10	1.15 <= version < 1.21	0.36.x	11.x	2019.5 <= version < 2021.4

[beckermr]

I think we need someone to add the patches now that the docs are official.

[jakirkham]

Please see PR ( conda-forge/conda-forge-repodata-patches-feedstock#376 ) for an attempt to add these to the repodata patches