Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Installing allennlp through pip using conda virtual environment fails #2473

Closed
ichn-hu opened this issue Feb 1, 2019 · 32 comments
Closed

Installing allennlp through pip using conda virtual environment fails #2473

ichn-hu opened this issue Feb 1, 2019 · 32 comments

Comments

@ichn-hu
Copy link

ichn-hu commented Feb 1, 2019

Describe the bug

I am trying to install allennlp through pip under conda virtual environment, however it fails and leave error message like this:

Building wheels for collected packages: overrides, jsonnet, nltk, parsimonious, numpydoc, msgpack, regex, ujson, dill, jsondiff, PyYAML, wrapt, cytoolz, future, toolz
  Running setup.py bdist_wheel for overrides ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/f7/27/b8/b4f46c59426a11e7f2d4e472b870ec14c21b4beab2e1afa725
  Running setup.py bdist_wheel for jsonnet ... error
  Complete output from command /home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/jsonnet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-t66ososi --python-tag cp37:
  running bdist_wheel
  running build
  running build_ext
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/desugarer.cpp -o core/desugarer.o
  core/desugarer.cpp: In member function ‘void Desugarer::desugar(AST*&, unsigned int)’:
  core/desugarer.cpp:612:51: warning: this statement may fall through [-Wimplicit-fallthrough=]
                   case BOP_MANIFEST_UNEQUAL: invert = true;
                                              ~~~~~~~^~~~~~
  core/desugarer.cpp:613:17: note: here
                   case BOP_MANIFEST_EQUAL: {
                   ^~~~
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/formatter.cpp -o core/formatter.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/libjsonnet.cpp -o core/libjsonnet.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/lexer.cpp -o core/lexer.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/parser.cpp -o core/parser.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/pass.cpp -o core/pass.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/static_analysis.cpp -o core/static_analysis.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/string_utils.cpp -o core/string_utils.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 core/vm.cpp -o core/vm.o
  g++ -c -g -O3 -Wall -Wextra -Woverloaded-virtual -pedantic -std=c++0x -fPIC -Iinclude -Ithird_party/md5 third_party/md5/md5.cpp -o third_party/md5/md5.o
  building '_jsonnet' extension
  creating build
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/python
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Iinclude -Ithird_party/md5 -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c python/_jsonnet.c -o build/temp.linux-x86_64-3.7/python/_jsonnet.o
  python/_jsonnet.c: In function ‘cpython_native_callback’:
  python/_jsonnet.c:147:19: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘const long unsigned int’} [-Wsign-compare]
       for (i = 0; i < ctx->argc; ++i) {
                     ^
  creating build/lib.linux-x86_64-3.7
  g++ -pthread -shared -B /home/ichn/anaconda3/envs/torch/compiler_compat -L/home/ichn/anaconda3/envs/torch/lib -Wl,-rpath=/home/ichn/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/python/_jsonnet.o core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o -o build/lib.linux-x86_64-3.7/_jsonnet.cpython-37m-x86_64-linux-gnu.so
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/python/_jsonnet.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'g++' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for jsonnet
  Running setup.py clean for jsonnet
  Running setup.py bdist_wheel for nltk ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/f1/98/72/c2ba4734bc46df30b9c3bd3eb037c52ab8ae0110f8fa15200a
  Running setup.py bdist_wheel for parsimonious ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/f1/a4/4b/7cac60fa74b7c16017cd9c67ab65736d3d9318064ae65e0ee0
  Running setup.py bdist_wheel for numpydoc ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/11/76/d4/16c19c2378616c3389916bc6d7b1134b72bfe6f7abd9f80243
  Running setup.py bdist_wheel for msgpack ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/3f/78/5a/92a8797deabe61189baf597a855e9529f6b20a391d9924d968
  Running setup.py bdist_wheel for regex ... error
  Complete output from command /home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/regex/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-6v75m_hn --python-tag cp37:
  /home/ichn/anaconda3/envs/torch/lib/python3.7/site-packages/setuptools/dist.py:470: UserWarning: Normalizing '2018.01.10' to '2018.1.10'
    normalized_version,
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  copying regex_3/regex.py -> build/lib.linux-x86_64-3.7
  copying regex_3/_regex_core.py -> build/lib.linux-x86_64-3.7
  copying regex_3/test_regex.py -> build/lib.linux-x86_64-3.7
  running build_ext
  building '_regex' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/regex_3
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.7/regex_3/_regex.o
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c regex_3/_regex_unicode.c -o build/temp.linux-x86_64-3.7/regex_3/_regex_unicode.o
  gcc -pthread -shared -B /home/ichn/anaconda3/envs/torch/compiler_compat -L/home/ichn/anaconda3/envs/torch/lib -Wl,-rpath=/home/ichn/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/regex_3/_regex.o build/temp.linux-x86_64-3.7/regex_3/_regex_unicode.o -o build/lib.linux-x86_64-3.7/_regex.cpython-37m-x86_64-linux-gnu.so
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/regex_3/_regex.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/regex_3/_regex.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/regex_3/_regex.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/regex_3/_regex.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/regex_3/_regex.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for regex
  Running setup.py clean for regex
  Running setup.py bdist_wheel for ujson ... error
  Complete output from command /home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-97buh6vk --python-tag cp37:
  Warning: 'classifiers' should be a list, got type 'filter'
  running bdist_wheel
  running build
  running build_ext
  building 'ujson' extension
  creating build
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/python
  creating build/temp.linux-x86_64-3.7/lib
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c ./python/ujson.c -o build/temp.linux-x86_64-3.7/./python/ujson.o -D_GNU_SOURCE
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c ./python/objToJSON.c -o build/temp.linux-x86_64-3.7/./python/objToJSON.o -D_GNU_SOURCE
  ./python/objToJSON.c: In function ‘PyUnicodeToUTF8’:
  ./python/objToJSON.c:154:18: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
       char *data = PyUnicode_AsUTF8AndSize(obj, &len);
                    ^~~~~~~~~~~~~~~~~~~~~~~
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c ./python/JSONtoObj.c -o build/temp.linux-x86_64-3.7/./python/JSONtoObj.o -D_GNU_SOURCE
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c ./lib/ultrajsonenc.c -o build/temp.linux-x86_64-3.7/./lib/ultrajsonenc.o -D_GNU_SOURCE
  ./lib/ultrajsonenc.c:156:23: warning: ‘g_hexChars’ is static but used in inline function ‘Buffer_AppendShortHexUnchecked’ which is not static
     *(outputOffset++) = g_hexChars[(value & 0x000f) >> 0];
                         ^~~~~~~~~~
  ./lib/ultrajsonenc.c:155:23: warning: ‘g_hexChars’ is static but used in inline function ‘Buffer_AppendShortHexUnchecked’ which is not static
     *(outputOffset++) = g_hexChars[(value & 0x00f0) >> 4];
                         ^~~~~~~~~~
  ./lib/ultrajsonenc.c:154:23: warning: ‘g_hexChars’ is static but used in inline function ‘Buffer_AppendShortHexUnchecked’ which is not static
     *(outputOffset++) = g_hexChars[(value & 0x0f00) >> 8];
                         ^~~~~~~~~~
  ./lib/ultrajsonenc.c:153:23: warning: ‘g_hexChars’ is static but used in inline function ‘Buffer_AppendShortHexUnchecked’ which is not static
     *(outputOffset++) = g_hexChars[(value & 0xf000) >> 12];
                         ^~~~~~~~~~
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./python -I./lib -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c ./lib/ultrajsondec.c -o build/temp.linux-x86_64-3.7/./lib/ultrajsondec.o -D_GNU_SOURCE
  creating build/lib.linux-x86_64-3.7
  gcc -pthread -shared -B /home/ichn/anaconda3/envs/torch/compiler_compat -L/home/ichn/anaconda3/envs/torch/lib -Wl,-rpath=/home/ichn/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/./python/ujson.o build/temp.linux-x86_64-3.7/./python/objToJSON.o build/temp.linux-x86_64-3.7/./python/JSONtoObj.o build/temp.linux-x86_64-3.7/./lib/ultrajsonenc.o build/temp.linux-x86_64-3.7/./lib/ultrajsondec.o -o build/lib.linux-x86_64-3.7/ujson.cpython-37m-x86_64-linux-gnu.so
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/./python/ujson.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/./python/ujson.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/./python/ujson.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/./python/ujson.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/./python/ujson.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for ujson
  Running setup.py clean for ujson
  Running setup.py bdist_wheel for dill ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/f6/d1/a7/c90dbb9c5613295c70d96d60e78c3e2b283143fddbbd57e14d
  Running setup.py bdist_wheel for jsondiff ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/8f/c9/36/f9e8aea16af567ce91abbe6b8b6b650877b9e17ce8aa97fb42
  Running setup.py bdist_wheel for PyYAML ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/11/c5/f8/4e054145468ca00fd2ab4a6c20bf7e09ec57b879572c865ee6
  Running setup.py bdist_wheel for wrapt ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/10/a6/59/eab55ff1e60d10ca0404baf6e7b8baf52908091133608bf289
  Running setup.py bdist_wheel for cytoolz ... error
  Complete output from command /home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/cytoolz/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ip_zgny_ --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/_signatures.py -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/__init__.py -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/utils_test.py -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/_version.py -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/compatibility.py -> build/lib.linux-x86_64-3.7/cytoolz
  creating build/lib.linux-x86_64-3.7/cytoolz/curried
  copying cytoolz/curried/operator.py -> build/lib.linux-x86_64-3.7/cytoolz/curried
  copying cytoolz/curried/__init__.py -> build/lib.linux-x86_64-3.7/cytoolz/curried
  copying cytoolz/curried/exceptions.py -> build/lib.linux-x86_64-3.7/cytoolz/curried
  copying cytoolz/dicttoolz.pyx -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/itertoolz.pyx -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/utils.pyx -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/recipes.pyx -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/functoolz.pyx -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/dicttoolz.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/__init__.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/recipes.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/utils.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/functoolz.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/itertoolz.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  copying cytoolz/cpython.pxd -> build/lib.linux-x86_64-3.7/cytoolz
  creating build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_none_safe.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_recipes.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_curried.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_tlz.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_itertoolz.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_functoolz.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/dev_skip_test.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_embedded_sigs.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_utils.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_docstrings.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_inspect_args.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_doctests.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_curried_toolzlike.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_serialization.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_compatibility.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_signatures.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_dev_skip_test.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  copying cytoolz/tests/test_dicttoolz.py -> build/lib.linux-x86_64-3.7/cytoolz/tests
  running build_ext
  building 'cytoolz.dicttoolz' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/cytoolz
  gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c cytoolz/dicttoolz.c -o build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o
  gcc -pthread -shared -B /home/ichn/anaconda3/envs/torch/compiler_compat -L/home/ichn/anaconda3/envs/torch/lib -Wl,-rpath=/home/ichn/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o -o build/lib.linux-x86_64-3.7/cytoolz/dicttoolz.cpython-37m-x86_64-linux-gnu.so
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o: unable to initialize decompress status for section .debug_info
  /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/cytoolz/dicttoolz.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for cytoolz
  Running setup.py clean for cytoolz
  Running setup.py bdist_wheel for future ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/3f/66/fe/9c4fd5c707a9f26993ba157f0752d84d5c7e26aedbefb84f76
  Running setup.py bdist_wheel for toolz ... done
  Stored in directory: /home/ichn/.cache/pip/wheels/73/ad/e1/f8fe78eeb9e2b31ea8396419d92adc107c553ff7eb47ad12d9
Successfully built overrides nltk parsimonious numpydoc msgpack dill jsondiff PyYAML wrapt future toolz
Failed to build jsonnet regex ujson cytoolz
Installing collected packages: overrides, jsonnet, wcwidth, ftfy, singledispatch, nltk, regex, murmurhash, ujson, cymem, dill, idna, chardet, urllib3, requests, msgpack, msgpack-numpy, tqdm, wrapt, preshed, toolz, cytoolz, plac, thinc, spacy, sqlparse, itsdangerous, click, werkzeug, MarkupSafe, Jinja2, flask, flask-cors, editdistance, flaky, cycler, pytz, kiwisolver, python-dateutil, pyparsing, matplotlib, greenlet, gevent, atomicwrites, py, pluggy, attrs, more-itertools, pytest, responses, jsonpickle, aws-xray-sdk, cookies, xmltodict, pbr, mock, jsondiff, jmespath, docutils, botocore, s3transfer, boto3, PyYAML, pyaml, boto, websocket-client, docker-pycreds, docker, asn1crypto, cryptography, future, ecdsa, pycryptodome, python-jose, moto, parsimonious, Pygments, alabaster, sphinxcontrib-websupport, imagesize, snowballstemmer, babel, packaging, sphinx, numpydoc, protobuf, tensorboardX, h5py, conllu, scipy, scikit-learn, unidecode, pytorch-pretrained-bert, colorama, pyasn1, rsa, awscli, allennlp
  Running setup.py install for jsonnet ... error
    Complete output from command /home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/jsonnet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-3ccckdjt/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    make: 'core/desugarer.o' is up to date.
    make: 'core/formatter.o' is up to date.
    make: 'core/libjsonnet.o' is up to date.
    make: 'core/lexer.o' is up to date.
    make: 'core/parser.o' is up to date.
    make: 'core/pass.o' is up to date.
    make: 'core/static_analysis.o' is up to date.
    make: 'core/string_utils.o' is up to date.
    make: 'core/vm.o' is up to date.
    make: 'third_party/md5/md5.o' is up to date.
    building '_jsonnet' extension
    creating build
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/python
    gcc -pthread -B /home/ichn/anaconda3/envs/torch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Iinclude -Ithird_party/md5 -I/home/ichn/anaconda3/envs/torch/include/python3.7m -c python/_jsonnet.c -o build/temp.linux-x86_64-3.7/python/_jsonnet.o
    python/_jsonnet.c: In function ‘cpython_native_callback’:
    python/_jsonnet.c:147:19: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘const long unsigned int’} [-Wsign-compare]
         for (i = 0; i < ctx->argc; ++i) {
                       ^
    creating build/lib.linux-x86_64-3.7
    g++ -pthread -shared -B /home/ichn/anaconda3/envs/torch/compiler_compat -L/home/ichn/anaconda3/envs/torch/lib -Wl,-rpath=/home/ichn/anaconda3/envs/torch/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/python/_jsonnet.o core/desugarer.o core/formatter.o core/libjsonnet.o core/lexer.o core/parser.o core/pass.o core/static_analysis.o core/string_utils.o core/vm.o third_party/md5/md5.o -o build/lib.linux-x86_64-3.7/_jsonnet.cpython-37m-x86_64-linux-gnu.so
    /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
    /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
    /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
    /home/ichn/anaconda3/envs/torch/compiler_compat/ld: build/temp.linux-x86_64-3.7/python/_jsonnet.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.7/python/_jsonnet.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    error: command 'g++' failed with exit status 1
    
    ----------------------------------------
Command "/home/ichn/anaconda3/envs/torch/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-grk1qblh/jsonnet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-3ccckdjt/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-grk1qblh/jsonnet/

To Reproduce

I am using conda 5.3.1 under archlinux with pytorch==1.0.0 preinstalled as part of the environment, and gcc of version

(torch) ➜  ~ g++ --version
g++ (GCC) 8.2.1 20181127
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

also tried install gcc through conda and then rerun

pip install allennlp

but fails as well.

Expected behavior

allennlp successfully installed

System (please complete the following information):

  • Linux
  • Python version: 3.7.2
  • AllenNLP version: I installed from master
  • PyTorch version: 1.0.0
@schmmd
Copy link
Member

schmmd commented Feb 5, 2019

I ran this on a mac, but I wasn't able to reproduce your issue:

$ conda create -n allennlp-3.7 python-3.7.2
$ conda activate allennlp-3.7
$ pip install allennlp
$ allennlp test-install

Our CI runs pip installs for each commit, although on an ubuntu instance and with Python 3.6.

@schmmd
Copy link
Member

schmmd commented Feb 6, 2019

Not sure if this helps:

$ gcc --version                                                                                                                                                                                                                                                                                       (allennlp-3.7) 
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I won't be able to troubleshoot the GCC problems you're having. Maybe you could try using our Docker image as a workaround?

@skywardpixel
Copy link

skywardpixel commented Feb 17, 2019 •
edited
Loading

Running into exactly the same issue on Manjaro Linux. This seems to be a compatibility issue with anaconda's linker (see cython/cython#2824) on Arch...

I got it working by renaming anaconda's ld to ld_ (following instructions here: pytorch/pytorch#16683) so the packages are linked with the OS's ld. Ugly solution, though.

@ichn-hu
Copy link
Author

ichn-hu commented Feb 19, 2019

I am also on Manjaro. My solution for this is to use conda-forge as the source channel of the installation.

@schmmd
Copy link
Member

schmmd commented Mar 15, 2019

Closing, since this has to do with a particular Linux distribution (Manjaro) and we have a workaround (see above two posts). Please comment further if there's a specific action you would like to see us take!

@schmmd schmmd closed this as completed Mar 15, 2019
@allanj
Copy link

allanj commented Mar 25, 2019

Reference to this issue: scrapy/scrapy#2115

sudo apt install python3.6-dev

This also works for me

@HenryDashwood
Copy link

HenryDashwood commented Mar 27, 2019 •
edited
Loading

This is also an issue on MacOS Mojave. The conda-forge workaround in #1983 does work but downgrades python 3.7 to 3.6 which is a little bit annoying.

@schmmd schmmd reopened this Mar 27, 2019
@schmmd
Copy link
Member

schmmd commented Mar 27, 2019

Re-opening since this might be a 3.7 issue.

@HenryDashwood
Copy link

HenryDashwood commented Mar 27, 2019 •
edited
Loading

pip install allennlp with python 3.6 does indeed work fine in Mojave.
But it does work with 3.7 when I do it in Ubuntu.

@vikigenius
Copy link
Contributor

I am facing the same issue. Works fine in 3.6 but fails if i use python 3.7. Tried in multiple distributions, Arch, and Void Linux, so it definitely looks like a python 3.7 issue

@schmmd
Copy link
Member

schmmd commented Apr 1, 2019

Interesting--we try out our library on python:3.7.2 (debian) for each commit to master.

@HenryDashwood
Copy link

HenryDashwood commented Apr 1, 2019 •
edited
Loading

I have another update re this issue. It fails when installing on an Amazon Linux AMI in both python 3.6 and python 3.7 conda environments.

@joelgrus
Copy link
Contributor

joelgrus commented Apr 1, 2019

I just started a brand new EC2 instance running the Amazon Linux 2 AMI. I was able to successfully install allennlp, here are the commands I ran:

sudo yum update
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod a+x Miniconda3-latest-Linux-x86_64.sh 
./Miniconda3-latest-Linux-x86_64.sh 
bash
conda create -n allennlp python=3.6
conda activate allennlp
sudo yum groupinstall "Development Tools"
pip install allennlp
allennlp train

@joelgrus
Copy link
Contributor

joelgrus commented Apr 1, 2019

on the same machine I was also able to install allennlp in a 3.7 environment:

conda create -n allennlp37 python=3.7
conda activate allennlp37
pip install allennlp
allennlp train

@HenryDashwood
Copy link

HenryDashwood commented Apr 1, 2019 •
edited
Loading

I'm so sorry Joel, but the error persists!
Here are the specific OS specs, if it helps:

NAME="Amazon Linux AMI"
VERSION="2018.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2018.03"
PRETTY_NAME="Amazon Linux AMI 2018.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

And here is the final part of the error message:

Command "/home/ec2-user/miniconda3/envs/allennlp37/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-061jz02j/jsonnet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))"install --record /tmp/pip-record-0mocpfuj/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-061jz02j/jsonnet/

The error message also begins with the line Failed building wheel for jsonnet.

@joelgrus
Copy link
Contributor

joelgrus commented Apr 1, 2019

ok, try doing gcc --version

on my "Amazon Linux 2" machine, it's 7.3.1. on my "Amazon Linux" machine, it's 7.2.1. so my hypothesis was that jsonnet needs 7.3.1 in order to build correctly. 😬

the question is: how to get the Amazon Linux machine up to 7.3.1. I was able to find these instructions, which worked (but also I have no idea what they're actually doing):

https://gist.github.com/mhart/841afde086604e7b4a86fa140480c84d

after running those, my gcc --version was showing 7.3.1 and I was able to pip install allennlp.

@HenryDashwood
Copy link

HenryDashwood commented Apr 1, 2019 •
edited
Loading

gcc is indeed version 7.2.1 on my AMI:
gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)

Unfortunately I won't be able to try 7.3.1 today as I don't have root access on this machine. But great debugging nonetheless! I also note that the gcc version in my conda env on macOS is gcc (GCC) 4.8.5

@joelgrus
Copy link
Contributor

joelgrus commented Apr 1, 2019

a simpler solution seems to be use the "Amazon Linux 2" AMI, but I get that that's not always an option.

@HenryDashwood
Copy link

Absolutely. Many thanks once again!

@vikigenius
Copy link
Contributor

I have tried to debug this more, and it seems to be an issue with jsonnet and regex packages. Both of them fail to install on conda environements with python 3.7 I followed @KaiChengYan 's suggestion and compile it against the system version of gcc and the system's ld. Once Jsonnet and Regex get installed I can just use pip install allennlp

@HenryDashwood
Copy link

HenryDashwood commented Apr 2, 2019 •
edited
Loading

I ran the gist today and it worked! Much appreciated!

@schmmd
Copy link
Member

schmmd commented Apr 11, 2019

I just had this issue on a Mac and followed https://stackoverflow.com/questions/52509602/cant-compile-c-program-on-a-mac-after-upgrade-to-mojave to get it to work. Specifically I ran:

https://stackoverflow.com/questions/52509602/cant-compile-c-program-on-a-mac-after-upgrade-to-mojave

@HenryDashwood
Copy link

How interesting. I updated to Mojave 10.14.4 (18E226) yesterday. I just tried to test this out in a new Conda environment, Python 3.7.3, but it installed perfectly on its own!

@schmmd
Copy link
Member

schmmd commented Apr 11, 2019

I was doing Conda, Python 3.6, on Mojave 10.14.4 (new laptop).

@DeNeutoy
Copy link
Contributor

Closing as resolved and dull

@Ravibth
Copy link

Ravibth commented Oct 18, 2019

I am getting below error in azure Linux environment when try to install allennlp.
Building wheel for jsonnet (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /opt/python/3.6.9/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-aog_zlna/jsonnet/s
etup.py'"'"'; file='"'"'/tmp/pip-install-aog_zlna/jsonnet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read
().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ncw4i672 --
python-tag cp36
cwd: /tmp/pip-install-aog_zlna/jsonnet/
Complete output (4 lines):
running bdist_wheel
running build
running build_ext
error: [Errno 2] No such file or directory: 'make': 'make'

ERROR: Failed building wheel for jsonnet

@schmmd
Copy link
Member

schmmd commented Oct 18, 2019

@Ravibth clearly you need to install make, which is already available on most systems. You might want to take a look at what we install on our base debian system in our Dockerfile. Please open a new issue if you have additional questions.

@Ravibth
Copy link

Ravibth commented Oct 22, 2019

Hi Michael,

I installed make. But Still I am getting same error.
image

When I use pip install jsonnet I am getting below
image

@manasRK
Copy link

manasRK commented Nov 12, 2019

I followed this on MacOC Mojave and was able to install AllenNLP.

https://stackoverflow.com/a/56380856/7572819

@schmmd
Copy link
Member

schmmd commented Nov 15, 2019

I plan to try conda install -c conda-forge jsonnet and I'm considering updating our instructions to make it the default.

@navidmdn
Copy link

navidmdn commented Aug 4, 2020

Reference to this issue: scrapy/scrapy#2115

sudo apt install python3.6-dev

This also works for me

Same worked on python 3.7 for me:

sudo apt install python3.7-dev

@thiborose
Copy link

I plan to try conda install -c conda-forge jsonnet and I'm considering updating our instructions to make it the default.

This one solved the problem on my machine (Ubuntu)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests