Numba 0.50.1 downloaded from conda fails to compile on GPU (but works when using pip)

[RemiLehe]

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included below a minimal working reproducer (if you are unsure how
  to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

Bug report

When installing numba from conda and running the standard test on GPU, i.e.

conda install numba
python -m numba.runtests numba.cuda.tests

I get a compilation error:

numba.cuda.cudadrv.error.NvvmError: Failed to compile

<unnamed> (25, 17): parse expected binary operation in atomicrmw
NVVM_ERROR_COMPILATION

On the other hand, when using pip:

pip install numba
python -m numba.runtests numba.cuda.tests

the tests are running without any issue.

When playing around (e.g. installing numba always from pip, but installing llvmlite either from conda or pip), it seemed to me that the issue comes from the version of llvmlite downloaded from conda...

More details

In the case of conda, the following packages were downloaded.

  llvmlite           pkgs/main/linux-64::llvmlite-0.33.0-py37hd408876_0
  numba              pkgs/main/linux-64::numba-0.50.1-py37h0573a6f_0
  tbb                pkgs/main/linux-64::tbb-2020.0-hfd86e86_0

On the other hand, in the case of pip, the following packages were downloaded:

llvmlite-0.33.0-cp37-cp37m-manylinux1_x86_64.whl
numba-0.50.1-cp37-cp37m-manylinux2014_x86_64.whl

The tests are run on a GTX 1080 Ti with cudatoolkit 9.0 installed from conda (Build h13b8566_0).

[stuartarchibald]

Thanks for the report, I can reproduce. From the error I think the Anaconda llvmdev builds aren't carrying the patch that prevents instruction upgrade.

[RemiLehe]

Thanks for looking into this.
Is there a chance that the conda build could be updated, so as to include the patch?

[seibert]

We're checking with them now to get a fixed build rolled out.

[stuartarchibald]

CC @angloyna for updates (with thanks for patching upstream!)

[angloyna]

Anaconda llvmdev has been patched, the latest numba 0.50.1 should have it. Let me know if we're still seeing the issue.

[RemiLehe]

Great! I confirm that this fixed the issue. Thanks a lot!

[stuartarchibald]

Thanks to you both @angloyna @RemiLehe, good to see this fixed.

[RemiLehe]

As a quick follow-up: I just noticed that, even though llvmlite 0.33 was updated in the numba channel, it is not updated in the conda-forge channel.
Therefore, users that were using e.g. conda install -c conda-forge numba were still getting the above-mentioned bug.

Is there any chance that the conda-forge package could be updated too?