Segfault when importing numba in conda environment #4515
Labels
Comments
|
Thanks for the report. I'm afraid I cannot replicate: this was on x86_64 Linux. Which glibc/libstdc++ etc do you have? Also, have you tried doing: or running under Thanks. |
|
Hi, thanks for the quick response. Here's my glibc version and my |
|
Thanks for the info, this: suggests there's something going wrong with the bindings to LLVM. Does this also segfault: ? |
|
Yes, that does also segfault. Here's the gdb backtrace for that as well - it's a bit shorter than the previous one: |
|
Thanks, this looks like a potential C++ ABI level problem, not sure what would be causing it as there's no reproducer yet. |
|
xref: Another user on gitter suggesting they can see the same https://gitter.im/numba/numba?at=5d8330b95d24375b3edffc67 |
|
Similarities, both users on Arch, both issues with |
|
I've managed to get around this by manually building llvmlite. I'm guessing somewhere in the pip/conda build process something went wrong. I cannot reproduce the error in a VM, but it's consistent in my system. |
|
@valorcurse thanks for the extra information, much appreciated. This matches up with the observation here too #4515 (comment). What was running in the VM? Thanks. |
|
The VM was running Manjaro 18.1.0, the same distro I'm running on my pc. |
|
I had the same issue with an install of numba from pypi into a conda environment (as a dependency of another package only on pypi). Installing llvmlite from conda, fixed the problem. |
This worked for me too. Just go |
|
Reproduced on a clean conda install on Arch Linux: Problem occurs even with llvmlite / numba==0.46 present. |
|
For anyone who runs into this problem, down-grading to conda-forge Python 3.6.7 avoids the issue for me. (Tested with Python 3.7.6 / numba 0.48 and the problem persists..) |
|
@khughitt thanks for submitting some additional information about this. Would you be able to submit a gdb backtrace of the segfault? Collecting backtraces from everyone who managed to reproduce this will hopefully allow us to identify a pattern to get closer to the root cause of this. Thanks! |
|
Sure thing! Here you go: Version info Just let me know if there is anything else I can help provide. |
|
@khughitt thanks! |
|
I just noticed the following: |
|
I also have this crash on Ubuntu 19.10. I created a new conda virtualenv with pyenv, and installed numba using pip. Here is the environment file. name: test-numba-pure
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- ca-certificates=2020.1.1=0
- certifi=2020.4.5.1=py38_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20181209=hc058e9b_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=9.1.0=hdf63c60_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- ncurses=6.2=he6710b0_0
- openssl=1.1.1g=h7b6447c_0
- pip=20.0.2=py38_1
- python=3.8.2=hcf32534_0
- readline=8.0=h7b6447c_0
- setuptools=46.1.3=py38_0
- sqlite=3.31.1=h62c20be_1
- tk=8.6.8=hbc83047_0
- wheel=0.34.2=py38_0
- xz=5.2.5=h7b6447c_0
- zlib=1.2.11=h7b6447c_3
- pip:
- llvmlite==0.32.0
- numba==0.49.0
- numpy==1.18.3
prefix: /home/angus/.pyenv/versions/miniconda3-latest/envs/test-numba-pureThe segfault does not occur if I install numba using conda: name: test-numba
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- blas=1.0=mkl
- ca-certificates=2020.1.1=0
- certifi=2020.4.5.1=py38_0
- intel-openmp=2020.0=166
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20181209=hc058e9b_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- llvmlite=0.32.0=py38hd408876_0
- mkl=2020.0=166
- mkl-service=2.3.0=py38he904b0f_0
- mkl_fft=1.0.15=py38ha843d7b_0
- mkl_random=1.1.0=py38h962f231_0
- ncurses=6.2=he6710b0_0
- numba=0.49.0=py38h0573a6f_0
- numpy=1.18.1=py38h4f9e942_0
- numpy-base=1.18.1=py38hde5b4d6_1
- openssl=1.1.1g=h7b6447c_0
- pip=20.0.2=py38_1
- python=3.8.2=hcf32534_0
- readline=8.0=h7b6447c_0
- setuptools=46.1.3=py38_0
- six=1.14.0=py38_0
- sqlite=3.31.1=h62c20be_1
- tbb=2020.0=hfd86e86_0
- tk=8.6.8=hbc83047_0
- wheel=0.34.2=py38_0
- xz=5.2.5=h7b6447c_0
- zlib=1.2.11=h7b6447c_3
prefix: /home/angus/.pyenv/versions/miniconda3-latest/envs/test-numba
|
|
@agoose77 thanks for this, I've built that env and unfortunately I cannot reproduce, am still suspicious it's related to glibc/libstdc++. Does this: segfault the same way? |
|
Yes, it does. Also, I can reproduce this for numba 0.48.0. For some reason, however, if i import PyROOT ( |
|
@agoose77 thanks. I'm pretty convinced it's down to DSO load ordering, #4515 (comment), reports similar. If you do: and then the same, but with |
|
No ROOT: ROOT: |
|
|
|
Yes, sorry - didn't realise the segfault didn't flush the file. |
|
@agoose77 thanks, from an initial look at this, it seems like ROOT is providing a LLVM toolchain and it might be simply that the symbols from llvmlite are resolving into that? |
|
Yes, thinking about it, ROOT uses llvm to facilitate CLING, so it's a good bet. |
Having the same issue on Debian 10. conda install llvmlite fixed the issue for me. Thanks! |
|
Hi, I have the same issue on my system:
As reported by others, if I install "numba" using Conda and do not create a virtual environment, everything works fine. However, I really need to use virtual environments for my work; in this case, something similar to what others have reported happens: |
stuartarchibald
added
bug - segfault
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
While trying to figure out what is causing lmcinnes/pynndescent#70, I've realized something very strange.
Importing numba causes a segfault:
However, this only happens in a conda environment. I can get this segfault consistently by setting up the environment like this:
However, if I set up the environment using
virtualenv, everything is fine:In both cases, the exact same packages are installed
Additionally,
llvmlitedoesn't seem to be the problem, since I can runpython -c 'import llvmlite'without segfaulting.At first, I suspected this had something to do with the new numpy version since it started failing at about the same time as the 1.17 release, but I can reproduce the segfault by downgrading numpy both to 1.16.4 and 1.15.4, so the issue must lie in numba somewhere.
I'm running Arch Linux x86_64 using Kernel: 5.2.11-arch1-1-ARCH.
The text was updated successfully, but these errors were encountered: