Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cython 0.29.2 & 0.29.4 of "Anaconda 2018.12" does not compatible with gcc 8.2.1 on Arch Linux #183

Closed
houchen-li opened this issue Feb 6, 2019 · 14 comments

Comments

@houchen-li
Copy link

I copied the code of fib.pyx from the tutorial and compiled it with Cython on my Arch Linux (gcc 8.2.1). Then I got the following error:

houchen_li@X411  /home/houchen_li/Projects/HerringModel   python setup.py build_ext --inplace
Compiling fib.pyx because it changed.
[1/1] Cythonizing fib.pyx
/opt/anaconda/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/houchen_li/Projects/HerringModel/fib.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'fib' extension
creating build
creating build/temp.linux-x86_64-3.7
gcc -pthread -B /opt/anaconda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda/include/python3.7m -c fib.c -o build/temp.linux-x86_64-3.7/fib.o
gcc -pthread -shared -B /opt/anaconda/compiler_compat -L/opt/anaconda/lib -Wl,-rpath=/opt/anaconda/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/fib.o -o /home/houchen_li/Projects/HerringModel/fib.cpython-37m-x86_64-linux-gnu.so
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/fib.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/fib.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/fib.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/fib.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-3.7/fib.o: file not recognized: file format not recognized
collect2: 错误:ld 返回 1
error: command 'gcc' failed with exit status 1

I guess it might because gcc 8.2.1 change to a new syntax so that Cython is not able to use gcc any more?
Does anyone have ideas?

cython/cython#2824 (comment)

@mingwandroid
Copy link

mingwandroid commented Feb 6, 2019 via email

@adrinjalali
Copy link

I've been struggling with the same issue, and haven't been able to find a nice documentation about it.

I guess something like a "prepare your build/development environment under conda" or something would be pretty useful.

Right now, my gcc and cython are from conda, but still I encounter the same issue pip installing scikit-learn's master.

error message:

    C compiler: gcc -pthread -B /home/adrin/.conda/envs/aif360/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC
    
    creating build/temp.linux-x86_64-3.6/sklearn/__check_build
    compile options: '-I/home/adrin/.conda/envs/aif360/lib/python3.6/site-packages/numpy/core/include -I/home/adrin/.conda/envs/aif360/lib/python3.6/site-packages/numpy/core/include -I/home/adrin/.conda/envs/aif360/include/python3.6m -c'
    gcc: sklearn/__check_build/_check_build.c
    gcc -pthread -shared -B /home/adrin/.conda/envs/aif360/compiler_compat -L/home/adrin/.conda/envs/aif360/lib -Wl,-rpath=/home/adrin/.conda/envs/aif360/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o -Lbuild/temp.linux-x86_64-3.6 -o build/lib.linux-x86_64-3.6/sklearn/__check_build/_check_build.cpython-36m-x86_64-linux-gnu.so
    /home/adrin/.conda/envs/aif360/compiler_compat/ld: build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: unable to initialize decompress status for section .debug_info
    /home/adrin/.conda/envs/aif360/compiler_compat/ld: build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    /home/adrin/.conda/envs/aif360/compiler_compat/ld: build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: unable to initialize decompress status for section .debug_info
    /home/adrin/.conda/envs/aif360/compiler_compat/ld: build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    error: Command "gcc -pthread -shared -B /home/adrin/.conda/envs/aif360/compiler_compat -L/home/adrin/.conda/envs/aif360/lib -Wl,-rpath=/home/adrin/.conda/envs/aif360/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/sklearn/__check_build/_check_build.o -Lbuild/temp.linux-x86_64-3.6 -o build/lib.linux-x86_64-3.6/sklearn/__check_build/_check_build.cpython-36m-x86_64-linux-gnu.so" failed with exit status 1
    
    ----------------------------------------
  Rolling back uninstall of scikit-learn
Command "/home/adrin/.conda/envs/aif360/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-2nygabxe/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-tsh82bp8/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-2nygabxe/

packages:

# Name                    Version                   Build  Channel
aif360-learn              0.1.dev1                  <pip>
alabaster                 0.7.12                    <pip>
AnyQt                     0.0.10                    <pip>
asn1crypto                0.24.0                    <pip>
asn1crypto                0.24.0                   py36_0  
atomicwrites              1.2.1                     <pip>
attrs                     18.2.0                    <pip>
autopep8                  1.4.3                     <pip>
backcall                  0.1.0                     <pip>
beautifulsoup4            4.6.3                    py36_0  
BlackBoxAuditing          0.1.40                    <pip>
blas                      1.0                         mkl  
bleach                    3.0.2                     <pip>
Bottleneck                1.2.1                     <pip>
bzip2                     1.0.6                h14c3975_5  
ca-certificates           2019.1.23                     0  
certifi                   2018.11.29               py36_0  
cffi                      1.11.5           py36he75722e_1  
chardet                   3.0.4                    py36_1  
chardet                   3.0.4                     <pip>
cloog                     0.18.0                        0  
cloudpickle               0.6.1                     <pip>
commonmark                0.8.1                     <pip>
conda                     4.5.12                   py36_0  
conda-build               3.17.6                   py36_0  
conda-env                 2.6.0                         1  
cryptography              2.4.1                     <pip>
cryptography              2.4.2            py36h1ba5d50_0  
cvxcanon                  0.1.1                    py36_0    cvxgrp
CVXcanon                  0.1.1                     <pip>
cvxpy                     1.0.6                    py36_0    cvxgrp
cycler                    0.10.0                    <pip>
cython                    0.29.4           py36he6710b0_0  
dask                      0.20.1                    <pip>
decorator                 4.3.0                     <pip>
defusedxml                0.5.0                     <pip>
dill                      0.2.8.2                  py36_0  
docutils                  0.14                      <pip>
ecos                      2.0.5                    py36_0    cvxgrp
ecos                      2.0.5                     <pip>
entrypoints               0.2.3                     <pip>
expat                     2.2.5             hf484d3e_1002    conda-forge
fastcache                 1.0.2                     <pip>
fastcache                 1.0.2            py36h14c3975_2  
filelock                  3.0.10                   py36_0  
flake8                    3.6.0                     <pip>
future                    0.17.1                    <pip>
gcc                       4.8.5                         7  
glob2                     0.6                      py36_1  
gmp                       6.1.2                h6c8ec71_1  
icu                       58.2                 h9c2bf20_1  
idna                      2.8                      py36_0  
idna                      2.7                       <pip>
imagesize                 1.1.0                     <pip>
intel-openmp              2019.1                      144  
ipykernel                 5.1.0                     <pip>
ipython                   7.1.1                     <pip>
ipython-genutils          0.2.0                     <pip>
ipywidgets                7.4.2                     <pip>
isl                       0.12.2                        0  
jedi                      0.13.1                    <pip>
jeepney                   0.4                       <pip>
jinja2                    2.10                     py36_0  
Jinja2                    2.10                      <pip>
joblib                    0.13.0                    <pip>
jsonschema                2.6.0                     <pip>
jupyter-client            5.2.3                     <pip>
jupyter-core              4.4.0                     <pip>
keyring                   16.0.2                    <pip>
keyrings.alt              3.1                       <pip>
kiwisolver                1.0.1                     <pip>
lapack                    3.6.1                ha44fe06_2    conda-forge
libarchive                3.3.3                h5d8350f_5  
libedit                   3.1.20170329         h6b74fdf_2  
libffi                    3.2.1                hd88cf55_4  
libgcc                    7.2.0                h69d50b8_2  
libgcc-ng                 8.2.0                hdf63c60_1  
libgfortran               3.0.0                         1    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0  
libiconv                  1.15              h14c3975_1004    conda-forge
liblief                   0.9.0                h7725739_1  
libstdcxx-ng              8.2.0                hdf63c60_1  
libxml2                   2.9.8                h26e45fe_1  
lime                      0.1.1.32                  <pip>
llvmlite                  0.25.0                    <pip>
lz4-c                     1.8.1.2              h14c3975_0  
lzo                       2.10                 h49e0be7_2  
markupsafe                1.1.0            py36h7b6447c_0  
MarkupSafe                1.1.0                     <pip>
matplotlib                3.0.2                     <pip>
mccabe                    0.6.1                     <pip>
mistune                   0.8.4                     <pip>
mkl                       2019.1                      144  
mkl_fft                   1.0.6            py36hd81dba3_0  
mkl_random                1.0.2            py36hd81dba3_0  
more-itertools            4.3.0                     <pip>
mpc                       1.0.3                hec55b23_5  
mpfr                      3.1.5                h11a74b3_2  
multiprocess              0.70.6.1                  <pip>
multiprocess              0.70.4                   py36_0    cvxgrp
nbconvert                 5.4.0                     <pip>
nbformat                  4.4.0                     <pip>
ncurses                   6.1                  hf484d3e_0  
networkx                  1.11                      <pip>
notebook                  5.7.0                     <pip>
numba                     0.40.1                    <pip>
numpy                     1.15.4           py36h7e9f1db_0  
numpy                     1.15.4                    <pip>
numpy-base                1.15.4           py36hde5b4d6_0  
openssl                   1.1.1a               h7b6447c_0  
Orange3                   3.7.1                     <pip>
osqp                      0.5.0                     <pip>
packaging                 19.0                      <pip>
pandas                    0.23.3                    <pip>
pandocfilters             1.4.2                     <pip>
parso                     0.3.1                     <pip>
patchelf                  0.9                  he6710b0_3  
pexpect                   4.6.0                     <pip>
pickleshare               0.7.5                     <pip>
Pillow                    5.3.0                     <pip>
pip                       18.1                     py36_0  
pip-upgrade-outdated      1.2                       <pip>
pkginfo                   1.4.2                    py36_1  
pluggy                    0.8.0                     <pip>
prometheus-client         0.4.2                     <pip>
prompt-toolkit            2.0.7                     <pip>
protobuf                  3.6.1                     <pip>
psutil                    5.4.8            py36h7b6447c_0  
ptyprocess                0.6.0                     <pip>
py                        1.7.0                     <pip>
py-lief                   0.9.0            py36h7725739_1  
pycodestyle               2.4.0                     <pip>
pycosat                   0.6.3            py36h14c3975_0  
pycparser                 2.19                     py36_0  
pycparser                 2.19                      <pip>
pyflakes                  2.0.0                     <pip>
Pygments                  2.2.0                     <pip>
pyopenssl                 18.0.0                   py36_0  
pyparsing                 2.3.0                     <pip>
pyqtgraph                 0.10.0                    <pip>
pysocks                   1.6.8                    py36_0  
pytest                    4.0.0                     <pip>
python                    3.6.7                h0371630_0  
python-dateutil           2.7.5                     <pip>
python-libarchive-c       2.8                      py36_6  
pytz                      2018.7                   py36_0  
PyWavelets                1.0.1                     <pip>
pyyaml                    3.13             py36h14c3975_0  
pyzmq                     17.1.2                    <pip>
readline                  7.0                  h7b6447c_5  
readme-renderer           24.0                      <pip>
requests                  2.20.1                    <pip>
requests                  2.21.0                   py36_0  
requests-toolbelt         0.8.0                     <pip>
ruamel_yaml               0.15.46          py36h14c3975_0  
scikit-image              0.14.1                    <pip>
scikit-learn              0.20.0                    <pip>
scipy                     1.1.0                     <pip>
scipy                     1.1.0            py36h7c811a0_2  
scs                       1.2.6                    py36_0    cvxgrp
scs                       2.0.2                     <pip>
SecretStorage             3.1.0                     <pip>
Send2Trash                1.5.0                     <pip>
serverfiles               0.2.1                     <pip>
setuptools                40.5.0                   py36_0  
six                       1.11.0                    <pip>
six                       1.12.0                   py36_0  
snowballstemmer           1.2.1                     <pip>
Sphinx                    1.8.3                     <pip>
sphinx-rtd-theme          0.4.2                     <pip>
sphinxcontrib-websupport  1.1.0                     <pip>
sqlite                    3.25.2               h7b6447c_0  
tensorflow                1.1.0                     <pip>
terminado                 0.8.1                     <pip>
testpath                  0.4.2                     <pip>
tk                        8.6.8                hbc83047_0  
toolz                     0.9.0                    py36_0  
toolz                     0.9.0                     <pip>
tornado                   5.1.1                     <pip>
tqdm                      4.28.1           py36h28b3542_0  
traitlets                 4.3.2                     <pip>
twine                     1.12.1                    <pip>
urllib3                   1.24.1                   py36_0  
urllib3                   1.24.1                    <pip>
wcwidth                   0.1.7                     <pip>
webencodings              0.5.1                     <pip>
Werkzeug                  0.14.1                    <pip>
wheel                     0.32.2                   py36_0  
widgetsnbextension        3.4.2                     <pip>
xlrd                      1.1.0                     <pip>
xz                        5.2.4                h14c3975_4  
yaml                      0.1.7                had09818_2  
zlib                      1.2.11               ha838bed_2  
zstd                      1.3.7                h0b5b093_0  

@houchen-li
Copy link
Author

I have no idea about this issue. In the end, I just stop using anaconda and switch to the system-wide cython. Then all goes fine!

@willprice
Copy link

I'm also coming up against this issue (also an Arch user). I presume there is some interaction going on with the host setup and the conda internals although I'm not quite sure where I should start digging. Any pointers and I can help debug this issue.

@mingwandroid
Copy link

mingwandroid commented Feb 12, 2019

The problem is that the system gcc is too new to work with the ld that we provide as part of our python package. We provide this to patch over link failures against our python libraries when using an older ld. Clearly we need something a bit more sophisticated. The problem all comes down to LTO information which while useful is compatible only with specific builds of GCC since they basically contain the state of the compiler. This kind of issue makes me want to switch away from GNU toolchains altogether.

The only thing you can do for now is to ensure our compilers are getting used. conda install gxx_linux-64 will do that, for example.

@willprice
Copy link

@mingwandroid Super, thanks for that workaround, that did the trick!

@mingwandroid
Copy link

It's not really a workaround as such, you are getting a more homogenous setup this way which is better, though these compilers don't look in system folders which is good for some use-cases but bad for others.

@willprice
Copy link

That's true. I was confused when I couldn't resolve the issue by conda install -c conda-forge gcc, I guess conda-build uses the defaults channel gcc if available?

@mingwandroid
Copy link

No one should use the old gcc packages. They're called things like gcc_linux-64 and clang_osx-64 instead. This is to allow cross-compilation.

Recipes should use the jinja2 functions that conda-build provides though when declaring dependencies on our compilers so e.g.:

requirements:
  build:
    - {{ compiler('c') }}

@willprice
Copy link

Thanks for the advice @mingwandroid

@hanslovsky
Copy link

Thanks, very helpful! If I wanted to supply a list of conda packages for people to install, how can I do this in a OS independent way, i.e. is there a OS agnostic package like gxx_linux-64 that picks the right OS specific package as needed?

@mcg1969
Copy link
Collaborator

mcg1969 commented Jul 3, 2019

I'm sorry I just noticed this, but this is actually in the wrong repo. This is the issue tracker for a very specific piece of software, not for the Anaconda Distribution in general.

@cpaulik
Copy link

cpaulik commented Aug 5, 2019

@mcg1969 Where should this issue go? I'm running into this on every new conda environment that I create and it is really annoying.

My workaround has been:

mv /home/cpa/.pyenv/versions/miniconda2-4.3.30/envs/vds_s1/compiler_compat/ld /home/cpa/.pyenv/versions/miniconda2-4.3.30/envs/vds_s1/compiler_compat/ld-old

@mcg1969
Copy link
Collaborator

mcg1969 commented Aug 5, 2019

Perhaps you intend this to be for https://github.com/ContinuumIO/anaconda-issues ? This is an issue tracker for a single, specific piece of software.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants