Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip install -r requirements.txt fails #3

Closed
RylanSchaeffer opened this issue Sep 29, 2020 · 9 comments
Closed

pip install -r requirements.txt fails #3

RylanSchaeffer opened this issue Sep 29, 2020 · 9 comments

Comments

@RylanSchaeffer
Copy link

I created a new virtual environment and immediately tried to install the requirements, but ran into a version error:

pip install -r requirements.txt 
Requirement 'jaxlib @ https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl' looks like a filename, but the file does not exist
Collecting asn1crypto==0.24.0 (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
    100% |████████████████████████████████| 102kB 2.9MB/s 
Collecting cloudpickle==1.2.2 (from -r requirements.txt (line 3))
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl
Collecting cryptography==2.1.4 (from -r requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/4e/e0/4959b48f04c879414972048fe2bedc96825e39c5413ae241c230fba58783/cryptography-2.1.4-cp36-cp36m-manylinux1_x86_64.whl (2.2MB)
    100% |████████████████████████████████| 2.2MB 721kB/s 
Collecting future==0.18.2 (from -r requirements.txt (line 5))
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)
    100% |████████████████████████████████| 829kB 1.6MB/s 
Collecting idna==2.6 (from -r requirements.txt (line 6))
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 5.9MB/s 
Collecting keyring==10.6.0 (from -r requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/b1/4a/89ab7aa2cf501a5e715c7bbb0df11af0c0b2b1d918cdfabca74984dd2c34/keyring-10.6.0-py2.py3-none-any.whl
Collecting keyrings.alt==3.0 (from -r requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/33/96/a2bc650d259ad510689656c50c06ae2de1299df59d369f21ec5ac0f20c8f/keyrings.alt-3.0-py2.py3-none-any.whl
Collecting opencv-python==4.2.0.34 (from -r requirements.txt (line 9))
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/72/c2/e9cf54ae5b1102020ef895866a67cb2e1aef72f16dd1fde5b5fb1495ad9c/opencv_python-4.2.0.34-cp36-cp36m-manylinux1_x86_64.whl
Collecting opt-einsum==3.2.1 (from -r requirements.txt (line 10))
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/63/a5/e6c07b08b934831ccb8c98ee335e66b7761c5754ee3cabfe4c11d0b1af28/opt_einsum-3.2.1-py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 5.8MB/s 
Collecting pycrypto==2.6.1 (from -r requirements.txt (line 11))
  Downloading https://files.pythonhosted.org/packages/60/db/645aa9af249f059cc3a368b118de33889219e0362141e75d4eaf6f80f163/pycrypto-2.6.1.tar.gz (446kB)
    100% |████████████████████████████████| 450kB 2.5MB/s 
Collecting pyglet==1.3.2 (from -r requirements.txt (line 12))
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/1c/fc/dad5eaaab68f0c21e2f906a94ddb98175662cc5a654eee404d59554ce0fa/pyglet-1.3.2-py2.py3-none-any.whl (1.0MB)
    100% |████████████████████████████████| 1.0MB 1.4MB/s 
Collecting pygobject==3.26.1 (from -r requirements.txt (line 13))
  Could not find a version that satisfies the requirement pygobject==3.26.1 (from -r requirements.txt (line 13)) (from versions: 3.27.0, 3.27.1, 3.27.2, 3.27.3, 3.27.4, 3.27.5, 3.28.0, 3.28.1, 3.28.2, 3.28.3, 3.29.1.dev0, 3.29.2.dev0, 3.29.3.dev0, 3.30.0, 3.30.1, 3.30.2, 3.30.3, 3.30.4, 3.30.5, 3.31.1.dev0, 3.31.2.dev0, 3.31.3.dev0, 3.31.4.dev0, 3.32.0, 3.32.1, 3.32.2, 3.33.1.dev0, 3.34.0, 3.36.0, 3.36.1, 3.38.0)
No matching distribution found for pygobject==3.26.1 (from -r requirements.txt (line 13))
@RylanSchaeffer
Copy link
Author

RylanSchaeffer commented Sep 29, 2020 •
edited

This is Python 3.6.9 and pip 20.2.3

@RylanSchaeffer
Copy link
Author

RylanSchaeffer commented Sep 29, 2020 •
edited

I tried removing the offending library pygobject and rerunning, but I ran into another error:

Collecting dm-haiku@ git+git://github.com/deepmind/dm-haiku.git@db991d56563221d5a06be5e7228155e53d01aba9
  Cloning git://github.com/deepmind/dm-haiku.git (to revision db991d56563221d5a06be5e7228155e53d01aba9) to /tmp/pip-install-h7datbt0/dm-haiku
    ERROR: Command errored out with exit status 1:
     command: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h7datbt0/dm-haiku/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h7datbt0/dm-haiku/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-uqzltyvu
         cwd: /tmp/pip-install-h7datbt0/dm-haiku/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-h7datbt0/dm-haiku/setup.py", line 18, in <module>
        from setuptools import find_namespace_packages
    ImportError: cannot import name 'find_namespace_packages'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

@RylanSchaeffer
Copy link
Author

RylanSchaeffer commented Sep 29, 2020 •
edited

A related error for dm-sonnet seemed to be caused by the user using Python 2 but I am certainly using Python3:

(dqn_zoo_venv) rylan@rylan-ThinkPad-E590:~/Documents/dqn_zoo$ pip --version
pip 20.2.3 from /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/lib/python3.6/site-packages/pip (python 3.6)

@RylanSchaeffer
Copy link
Author

Another post suggested uninstalling and then reinstalling setuptools, which worked for me.

@RylanSchaeffer
Copy link
Author

But now a new error appears with python-snappy:

~/Documents/dqn_zoo$ pip install -r requirements.txt 
Collecting asn1crypto==0.24.0
  Using cached asn1crypto-0.24.0-py2.py3-none-any.whl (101 kB)
Collecting cloudpickle==1.2.2
  Using cached cloudpickle-1.2.2-py2.py3-none-any.whl (25 kB)
Collecting cryptography==2.1.4
  Using cached cryptography-2.1.4-cp36-cp36m-manylinux1_x86_64.whl (2.2 MB)
Collecting future==0.18.2
  Using cached future-0.18.2.tar.gz (829 kB)
Collecting idna==2.6
  Using cached idna-2.6-py2.py3-none-any.whl (56 kB)
Collecting keyring==10.6.0
  Using cached keyring-10.6.0-py2.py3-none-any.whl (38 kB)
Collecting keyrings.alt==3.0
  Using cached keyrings.alt-3.0-py2.py3-none-any.whl (21 kB)
Collecting opencv-python==4.2.0.34
  Using cached opencv_python-4.2.0.34-cp36-cp36m-manylinux1_x86_64.whl (28.2 MB)
Collecting opt-einsum==3.2.1
  Using cached opt_einsum-3.2.1-py3-none-any.whl (63 kB)
Collecting pycrypto==2.6.1
  Using cached pycrypto-2.6.1.tar.gz (446 kB)
Collecting pyglet==1.3.2
  Using cached pyglet-1.3.2-py2.py3-none-any.whl (1.0 MB)
Collecting pyxdg==0.25
  Using cached pyxdg-0.25.tar.gz (48 kB)
Collecting SecretStorage==2.3.1
  Using cached SecretStorage-2.3.1.tar.gz (16 kB)
Collecting six==1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting absl-py==0.9.0
  Using cached absl-py-0.9.0.tar.gz (104 kB)
Collecting numpy==1.18.0
  Using cached numpy-1.18.0-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB)
Collecting Pillow==7.1.2
  Using cached Pillow-7.1.2-cp36-cp36m-manylinux1_x86_64.whl (2.1 MB)
Collecting python-snappy==0.5.4
  Using cached python-snappy-0.5.4.tar.gz (21 kB)
Collecting scipy==1.2.1
  Using cached scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8 MB)
Processing /home/rylan/.cache/pip/wheels/af/49/6c/0e95549f4da2ae98ecdb63d3873a11b6b757151e0ae0ae1b9f/jax-0.1.72-py3-none-any.whl
Collecting jaxlib@ https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl
  Using cached https://storage.googleapis.com/jax-releases/cuda101/jaxlib-0.1.49-cp36-none-linux_x86_64.whl (62.0 MB)
Collecting atari-py==0.2.6
  Using cached atari_py-0.2.6-cp36-cp36m-manylinux1_x86_64.whl (2.8 MB)
Collecting gym==0.13.1
  Using cached gym-0.13.1.tar.gz (1.6 MB)
Collecting dm-env==1.2
  Using cached dm_env-1.2-py3-none-any.whl (22 kB)
Collecting dm-haiku@ git+git://github.com/deepmind/dm-haiku.git@db991d56563221d5a06be5e7228155e53d01aba9
  Cloning git://github.com/deepmind/dm-haiku.git (to revision db991d56563221d5a06be5e7228155e53d01aba9) to /tmp/pip-install-3c8l7h9u/dm-haiku
Collecting dm-tree==0.1.5
  Using cached dm_tree-0.1.5-cp36-cp36m-manylinux1_x86_64.whl (294 kB)
Collecting optax==0.0.1
  Downloading optax-0.0.1-py3-none-any.whl (49 kB)
     |████████████████████████████████| 49 kB 1.8 MB/s 
Collecting rlax@ git+git://github.com/deepmind/rlax.git@870cba1ea8ad36725f4f3a790846298657b6fd4b
  Cloning git://github.com/deepmind/rlax.git (to revision 870cba1ea8ad36725f4f3a790846298657b6fd4b) to /tmp/pip-install-3c8l7h9u/rlax
Collecting cffi>=1.7; platform_python_implementation != "PyPy"
  Downloading cffi-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (400 kB)
     |████████████████████████████████| 400 kB 6.9 MB/s 
Collecting chex>=0.0.2
  Downloading chex-0.0.2-py3-none-any.whl (37 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting dataclasses==0.7; python_version < "3.7"
  Using cached dataclasses-0.7-py3-none-any.whl (18 kB)
Collecting toolz>=0.9.0
  Downloading toolz-0.11.1-py3-none-any.whl (55 kB)
     |████████████████████████████████| 55 kB 4.6 MB/s 
Using legacy 'setup.py install' for future, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pycrypto, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pyxdg, since package 'wheel' is not installed.
Using legacy 'setup.py install' for SecretStorage, since package 'wheel' is not installed.
Using legacy 'setup.py install' for absl-py, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-snappy, since package 'wheel' is not installed.
Using legacy 'setup.py install' for gym, since package 'wheel' is not installed.
Using legacy 'setup.py install' for dm-haiku, since package 'wheel' is not installed.
Using legacy 'setup.py install' for rlax, since package 'wheel' is not installed.
Installing collected packages: asn1crypto, cloudpickle, six, idna, pycparser, cffi, cryptography, future, SecretStorage, keyring, keyrings.alt, numpy, opencv-python, opt-einsum, pycrypto, pyglet, pyxdg, absl-py, Pillow, python-snappy, scipy, jax, jaxlib, atari-py, gym, dm-tree, dm-env, dm-haiku, dataclasses, toolz, chex, optax, rlax
    Running setup.py install for future ... done
    Running setup.py install for SecretStorage ... done
    Running setup.py install for pycrypto ... done
    Running setup.py install for pyxdg ... done
    Running setup.py install for absl-py ... done
    Running setup.py install for python-snappy ... error
    ERROR: Command errored out with exit status 1:
     command: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5vwf1ixf/install-record.txt --single-version-externally-managed --compile --install-headers /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include/site/python3.6/python-snappy
         cwd: /tmp/pip-install-3c8l7h9u/python-snappy/
    Complete output (23 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/snappy
    copying snappy/__main__.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/snappy.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/__init__.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-3.6/snappy
    copying snappy/snappy_formats.py -> build/lib.linux-x86_64-3.6/snappy
    running build_ext
    building 'snappy._snappy' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/snappy
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include -I/usr/include/python3.6m -c snappy/snappymodule.cc -o build/temp.linux-x86_64-3.6/snappy/snappymodule.o
    snappy/snappymodule.cc:31:10: fatal error: snappy-c.h: No such file or directory
     #include <snappy-c.h>
              ^~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3c8l7h9u/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5vwf1ixf/install-record.txt --single-version-externally-managed --compile --install-headers /home/rylan/Documents/dqn_zoo/dqn_zoo_venv/include/site/python3.6/python-snappy Check the logs for full command output.

@RylanSchaeffer
Copy link
Author

If you have an error with python-snappy, you may need to install Snappy C separately

https://stackoverflow.com/questions/11416024/error-installing-python-snappy-snappy-c-h-no-such-file-or-directory

@RylanSchaeffer
Copy link
Author

Ok everything succeeded so long as pygobject==3.26.1 is omitted.

@jqdm
Copy link
Collaborator

jqdm commented Oct 1, 2020

I've clarified the purpose of requirements.txt in f6ea040. These kind of dependency issues are very common since everyone has a different set up and is precisely why a Dockerfile was provided. requirements.txt is specifically only for use with Dockerfile, and is not generally applicable.

@jqdm jqdm closed this as completed Oct 1, 2020
@Zekzak31
Copy link

python3 -m pip install-r requirements.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@RylanSchaeffer @jqdm @Zekzak31