Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typing-extensions package is conflicted #51743

Closed
Taehun opened this issue Aug 30, 2021 · 22 comments
Closed

typing-extensions package is conflicted #51743

Taehun opened this issue Aug 30, 2021 · 22 comments
Assignees
Labels
2.6.0 stat:awaiting tensorflower Status - Awaiting response from tensorflower subtype:macOS macOS Build/Installation issues type:build/install Build and install issues

Comments

@Taehun
Copy link
Contributor

Taehun commented Aug 30, 2021

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS 11.5.2
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: -
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version: 2.6.0
  • Python version: 3.8.10
  • Installed using virtualenv? pip? conda?: virtualenv and pip
  • Bazel version (if compiling from source): -
  • GCC/Compiler version (if compiling from source): -
  • CUDA/cuDNN version: -
  • GPU model and memory: -

Describe the problem
I installed tensorflow using pip on the virtualenv. The typing-extensions package is conflicted with black package dependency. black==21.8b0 is required typing-extensions>=3.10.0.0. But, tensorflow has 'typing_extensions ~= 3.7.4 dependency. Are there any plans to upgrade the typing-extensions dependency package version?

Provide the exact sequence of commands / steps that you executed before running into the problem

$ python3 -m venv venv
$ . venv/bin/activate
$ pip install -U pip
$ pip install black
$ pip install tensorflow

Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

$ pip install tensorflow
Requirement already satisfied: tensorflow in ./venv/lib/python3.8/site-packages (2.6.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.1.2)
Requirement already satisfied: astunparse~=1.6.3 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.6.3)
Requirement already satisfied: opt-einsum~=3.3.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.3.0)
Collecting typing-extensions~=3.7.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Requirement already satisfied: numpy~=1.19.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.19.5)
Requirement already satisfied: wrapt~=1.12.1 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.12.1)
Requirement already satisfied: tensorflow-estimator~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: six~=1.15.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.15.0)
Requirement already satisfied: wheel~=0.35 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.37.0)
Requirement already satisfied: google-pasta~=0.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.2.0)
Requirement already satisfied: flatbuffers~=1.12.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.12)
Requirement already satisfied: tensorboard~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: keras~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: protobuf>=3.9.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.17.3)
Requirement already satisfied: gast==0.4.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.4.0)
Requirement already satisfied: termcolor~=1.1.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.1.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.39.0)
Requirement already satisfied: clang~=5.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (5.0)
Requirement already satisfied: absl-py~=0.10 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.13.0)
Requirement already satisfied: h5py~=3.1.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.1.0)
Requirement already satisfied: markdown>=2.6.8 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (3.3.4)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.8.0)
Requirement already satisfied: werkzeug>=0.11.15 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (0.6.1)
Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (56.0.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (0.4.5)
Requirement already satisfied: requests<3,>=2.21.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (2.26.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (4.2.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (4.7.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in ./venv/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow) (1.3.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in ./venv/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (0.4.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (1.26.5)
Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (3.2)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (2.0.4)
Requirement already satisfied: oauthlib>=3.0.0 in ./venv/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow) (3.1.1)
Installing collected packages: typing-extensions
  Attempting uninstall: typing-extensions
    Found existing installation: typing-extensions 3.10.0.1
    Uninstalling typing-extensions-3.10.0.1:
      Successfully uninstalled typing-extensions-3.10.0.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
black 21.8b0 requires typing-extensions>=3.10.0.0, but you have typing-extensions 3.7.4.3 which is incompatible.
Successfully installed typing-extensions-3.7.4.3
@Taehun Taehun added the type:build/install Build and install issues label Aug 30, 2021
@mohantym mohantym added 2.6.0 subtype:macOS macOS Build/Installation issues labels Aug 30, 2021
@mohantym
Copy link
Contributor

Hi @sanatmpa1 ,Could you please look into this issue?

@matthewfeickert
Copy link

matthewfeickert commented Sep 6, 2021

If just the two are specified then pip is able to figure out the dependency mismatch and find a version of black (v21.7.b0) that will work

Full Example in a python:3.9 Docker image:
$ docker run --rm -ti python:3.9 /bin/bash
root@12adf30cd4d4:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@12adf30cd4d4:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@12adf30cd4d4:/# python -m pip install black tensorflow
Collecting black
  Downloading black-21.8b0-py3-none-any.whl (148 kB)
     |████████████████████████████████| 148 kB 6.3 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.6.0-cp39-cp39-manylinux2010_x86_64.whl (458.4 MB)
     |████████████████████████████████| 458.4 MB 40.8 MB/s 
Collecting pathspec<1,>=0.9.0
  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting tomli<2.0.0,>=0.2.6
  Downloading tomli-1.2.1-py3-none-any.whl (11 kB)
Collecting click>=7.1.2
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 9.5 MB/s 
Collecting mypy-extensions>=0.4.3
  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting platformdirs>=2
  Downloading platformdirs-2.3.0-py3-none-any.whl (13 kB)
Collecting typing-extensions>=3.10.0.0
  Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting regex>=2020.1.8
  Downloading regex-2021.8.28-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (759 kB)
     |████████████████████████████████| 759 kB 42.1 MB/s 
Collecting keras~=2.6
  Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 51.1 MB/s 
Collecting clang~=5.0
  Downloading clang-5.0.tar.gz (30 kB)
Collecting h5py~=3.1.0
  Downloading h5py-3.1.0-cp39-cp39-manylinux1_x86_64.whl (4.4 MB)
     |████████████████████████████████| 4.4 MB 59.1 MB/s 
Collecting keras-preprocessing~=1.1.2
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 2.4 MB/s 
Collecting opt-einsum~=3.3.0
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 5.3 MB/s 
Collecting grpcio<2.0,>=1.37.0
  Downloading grpcio-1.39.0-cp39-cp39-manylinux2014_x86_64.whl (4.3 MB)
     |████████████████████████████████| 4.3 MB 31.8 MB/s 
Collecting flatbuffers~=1.12.0
  Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting six~=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting gast==0.4.0
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting tensorboard~=2.6
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 29.8 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.5.1-cp39-cp39-manylinux2010_x86_64.whl (454.5 MB)
     |████████████████████████████████| 454.5 MB 23.6 MB/s 
Collecting grpcio~=1.34.0
  Downloading grpcio-1.34.1-cp39-cp39-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 38.4 MB/s 
Collecting keras-nightly~=2.5.0.dev
  Downloading keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 72.1 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.5.0-cp39-cp39-manylinux2010_x86_64.whl (454.4 MB)
     |████████████████████████████████| 454.4 MB 73 kB/s 
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of black to determine which version is compatible with other requirements. This could take a while.
Collecting black
  Downloading black-21.7b0-py3-none-any.whl (141 kB)
     |████████████████████████████████| 141 kB 52.8 MB/s 
Collecting appdirs
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting typing-extensions~=3.7.4
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting termcolor~=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting astunparse~=1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting numpy~=1.19.2
  Downloading numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB 41.4 MB/s 
Collecting absl-py~=0.10
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 28.9 MB/s 
Collecting tensorflow-estimator~=2.6
  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 29.1 MB/s 
Requirement already satisfied: wheel~=0.35 in /usr/local/venv/lib/python3.9/site-packages (from tensorflow) (0.37.0)
Collecting wrapt~=1.12.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting protobuf>=3.9.2
  Downloading protobuf-3.17.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
     |████████████████████████████████| 1.0 MB 34.8 MB/s 
Collecting google-pasta~=0.2
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 8.0 MB/s 
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 40.0 MB/s 
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
     |████████████████████████████████| 4.9 MB 32.5 MB/s 
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/venv/lib/python3.9/site-packages (from tensorboard~=2.6->tensorflow) (58.0.2)
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 32.8 MB/s 
Collecting requests<3,>=2.21.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 1.2 MB/s 
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 29.1 MB/s 
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 10.2 MB/s 
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 32.7 MB/s 
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 8.6 MB/s 
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 37.6 MB/s 
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 10.0 MB/s 
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 28.4 MB/s 
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 26.5 MB/s 
Building wheels for collected packages: clang, termcolor, wrapt
  Building wheel for clang (setup.py) ... done
  Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30692 sha256=26573dc9851cb201eebc9f3acd403834783398b5f7140b7fd86d1942a44c38aa
  Stored in directory: /root/.cache/pip/wheels/3a/ce/7a/27094f689461801c934296d07078773603663dfcaca63bb064
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4847 sha256=e374182ff99d7aa8fff47b8b498e27a1a475777ed7c2ecad2abc722f62b221bf
  Stored in directory: /root/.cache/pip/wheels/b6/0d/90/0d1bbd99855f99cb2f6c2e5ff96f8023fad8ec367695f7d72d
  Building wheel for wrapt (setup.py) ... done
  Created wheel for wrapt: filename=wrapt-1.12.1-cp39-cp39-linux_x86_64.whl size=73034 sha256=92b09a8a10deab562a852acbbfe5c7cda670f0e4c1c47103f2a0591854ec8fa7
  Stored in directory: /root/.cache/pip/wheels/98/23/68/efe259aaca055e93b08e74fbe512819c69a2155c11ba3c0f10
Successfully built clang termcolor wrapt
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, werkzeug, tensorboard-plugin-wit, tensorboard-data-server, protobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, typing-extensions, tomli, termcolor, tensorflow-estimator, tensorboard, regex, pathspec, opt-einsum, mypy-extensions, keras-preprocessing, keras, h5py, google-pasta, gast, flatbuffers, click, clang, astunparse, appdirs, tensorflow, black
Successfully installed absl-py-0.13.0 appdirs-1.4.4 astunparse-1.6.3 black-21.7b0 cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.4 clang-5.0 click-8.0.1 flatbuffers-1.12 gast-0.4.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.39.0 h5py-3.1.0 idna-3.2 keras-2.6.0 keras-preprocessing-1.1.2 markdown-3.3.4 mypy-extensions-0.4.3 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 pathspec-0.9.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 regex-2021.8.28 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.15.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-2.6.0 tensorflow-estimator-2.6.0 termcolor-1.1.0 tomli-1.2.1 typing-extensions-3.7.4.3 urllib3-1.26.6 werkzeug-2.0.1 wrapt-1.12.1
(venv) root@12adf30cd4d4:/# pip list | grep "black\|tensorflow"
black                   21.7b0
tensorflow              2.6.0
tensorflow-estimator    2.6.0

but that tensorflow is pinning down to the patch release here (typing-extensions~=3.7.4) is pretty restrictive (and making it so that everyone using TensorFlow needs to treat it like an application and not a library. c.f. PR #40789 for another example of problems coming from being overly restrictive, though this is just a historical note as the SciPy dependency has been removed and so no longer affects TensorFlow).

In more complicated environments though I can get pip to fail on this problem.

Edit: Following @alexander-held example below I can get things to install on Python 3.9 but not on Python 3.8. c.f. example in details for pip install black tensorflow tensorflow-probability on Python 3.9

Example with TensorFlow Probability in a python:3.9 Docker image:
$ docker run --rm -ti python:3.9 /bin/bash
root@320d8517e990:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@320d8517e990:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@320d8517e990:/# python -m pip install black tensorflow tensorflow-probability
Collecting black
  Downloading black-21.8b0-py3-none-any.whl (148 kB)
     |████████████████████████████████| 148 kB 5.6 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.6.0-cp39-cp39-manylinux2010_x86_64.whl (458.4 MB)
     |████████████████████████████████| 458.4 MB 28 kB/s 
Collecting tensorflow-probability
  Downloading tensorflow_probability-0.13.0-py2.py3-none-any.whl (5.4 MB)
     |████████████████████████████████| 5.4 MB 46.2 MB/s 
Collecting mypy-extensions>=0.4.3
  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting tomli<2.0.0,>=0.2.6
  Downloading tomli-1.2.1-py3-none-any.whl (11 kB)
Collecting platformdirs>=2
  Downloading platformdirs-2.3.0-py3-none-any.whl (13 kB)
Collecting typing-extensions>=3.10.0.0
  Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting regex>=2020.1.8
  Downloading regex-2021.8.28-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (759 kB)
     |████████████████████████████████| 759 kB 33.1 MB/s 
Collecting click>=7.1.2
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 9.6 MB/s 
Collecting pathspec<1,>=0.9.0
  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting clang~=5.0
  Downloading clang-5.0.tar.gz (30 kB)
Collecting astunparse~=1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting wrapt~=1.12.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting keras-preprocessing~=1.1.2
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 1.9 MB/s 
Collecting grpcio<2.0,>=1.37.0
  Downloading grpcio-1.39.0-cp39-cp39-manylinux2014_x86_64.whl (4.3 MB)
     |████████████████████████████████| 4.3 MB 41.9 MB/s 
Collecting protobuf>=3.9.2
  Downloading protobuf-3.17.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
     |████████████████████████████████| 1.0 MB 47.2 MB/s 
Collecting google-pasta~=0.2
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 7.9 MB/s 
Collecting tensorflow-estimator~=2.6
  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 56.3 MB/s 
Requirement already satisfied: wheel~=0.35 in /usr/local/venv/lib/python3.9/site-packages (from tensorflow) (0.37.0)
Collecting tensorboard~=2.6
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 58.4 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.5.1-cp39-cp39-manylinux2010_x86_64.whl (454.5 MB)
     |████████████████████████████████| 454.5 MB 90 kB/s 
Collecting tensorflow-estimator<2.6.0,>=2.5.0
  Downloading tensorflow_estimator-2.5.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 54.2 MB/s 
Collecting tensorflow
  Downloading tensorflow-2.5.0-cp39-cp39-manylinux2010_x86_64.whl (454.4 MB)
     |████████████████████████████████| 454.4 MB 65 kB/s 
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of black to determine which version is compatible with other requirements. This could take a while.
Collecting black
  Downloading black-21.7b0-py3-none-any.whl (141 kB)
     |████████████████████████████████| 141 kB 45.6 MB/s 
Collecting appdirs
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting typing-extensions~=3.7.4
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting numpy~=1.19.2
  Downloading numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB 40.7 MB/s 
Collecting flatbuffers~=1.12.0
  Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting h5py~=3.1.0
  Downloading h5py-3.1.0-cp39-cp39-manylinux1_x86_64.whl (4.4 MB)
     |████████████████████████████████| 4.4 MB 37.6 MB/s 
Collecting six~=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting termcolor~=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting opt-einsum~=3.3.0
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 6.1 MB/s 
Collecting keras~=2.6
  Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 40.4 MB/s 
Collecting gast==0.4.0
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting absl-py~=0.10
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 26.6 MB/s 
Collecting decorator
  Downloading decorator-5.0.9-py3-none-any.whl (8.9 kB)
Collecting cloudpickle>=1.3
  Downloading cloudpickle-1.6.0-py3-none-any.whl (23 kB)
Collecting dm-tree
  Downloading dm_tree-0.1.6-cp39-cp39-manylinux_2_24_x86_64.whl (94 kB)
     |████████████████████████████████| 94 kB 4.5 MB/s 
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 45.8 MB/s 
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
     |████████████████████████████████| 4.9 MB 36.4 MB/s 
Collecting requests<3,>=2.21.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 855 kB/s 
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/venv/lib/python3.9/site-packages (from tensorboard~=2.6->tensorflow) (58.0.2)
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 54.4 MB/s 
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 55.7 MB/s 
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 9.9 MB/s 
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 52.0 MB/s 
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 6.6 MB/s 
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 12.8 MB/s 
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 54.1 MB/s 
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 38.8 MB/s 
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 50.2 MB/s 
Building wheels for collected packages: clang, termcolor, wrapt
  Building wheel for clang (setup.py) ... done
  Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30692 sha256=90a2386782700fa4c0e5b920347a2421b9ae6ba24b65aeb409d798a9fd084506
  Stored in directory: /root/.cache/pip/wheels/3a/ce/7a/27094f689461801c934296d07078773603663dfcaca63bb064
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4847 sha256=800d922b6de838ad1211e5a07fa912cffbc8783c2564bb0f55a31068cffa99c5
  Stored in directory: /root/.cache/pip/wheels/b6/0d/90/0d1bbd99855f99cb2f6c2e5ff96f8023fad8ec367695f7d72d
  Building wheel for wrapt (setup.py) ... done
  Created wheel for wrapt: filename=wrapt-1.12.1-cp39-cp39-linux_x86_64.whl size=73028 sha256=f484ce3455daa6aeb299cda0a61d540b6741d8088e99edc9cf7fb5d49ac8c8cc
  Stored in directory: /root/.cache/pip/wheels/98/23/68/efe259aaca055e93b08e74fbe512819c69a2155c11ba3c0f10
Successfully built clang termcolor wrapt
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, werkzeug, tensorboard-plugin-wit, tensorboard-data-server, protobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, typing-extensions, tomli, termcolor, tensorflow-estimator, tensorboard, regex, pathspec, opt-einsum, mypy-extensions, keras-preprocessing, keras, h5py, google-pasta, gast, flatbuffers, dm-tree, decorator, cloudpickle, click, clang, astunparse, appdirs, tensorflow-probability, tensorflow, black
Successfully installed absl-py-0.13.0 appdirs-1.4.4 astunparse-1.6.3 black-21.7b0 cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.4 clang-5.0 click-8.0.1 cloudpickle-1.6.0 decorator-5.0.9 dm-tree-0.1.6 flatbuffers-1.12 gast-0.4.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.39.0 h5py-3.1.0 idna-3.2 keras-2.6.0 keras-preprocessing-1.1.2 markdown-3.3.4 mypy-extensions-0.4.3 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 pathspec-0.9.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 regex-2021.8.28 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.15.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-2.6.0 tensorflow-estimator-2.6.0 tensorflow-probability-0.13.0 termcolor-1.1.0 tomli-1.2.1 typing-extensions-3.7.4.3 urllib3-1.26.6 werkzeug-2.0.1 wrapt-1.12.1
(venv) root@320d8517e990:/# pip list | grep "black\|tensorflow"
black                   21.7b0
tensorflow              2.6.0
tensorflow-estimator    2.6.0
tensorflow-probability  0.13.0

@alexander-held
Copy link

An example for a setup where the dependency resolution fails is the combination of black, tensorflow and tensorflow-probability. Using a python:3.8 Docker environment (otherwise equivalent setup to @matthewfeickert's comment above),

python -m pip install black tensorflow tensorflow-probability

installs the following (output of pip list | grep "black\|tensorflow"):

black                   21.8b0
tensorflow              2.3.4
tensorflow-estimator    2.3.0
tensorflow-probability  0.13.0

The import of tensorflow-probability after tensorflow then fails:

python -c "import tensorflow, tensorflow_probability"

results in

ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.5; Detected an installation of version 2.3.4. Please upgrade TensorFlow to proceed.

@mihaimaruseac
Copy link
Collaborator

but that tensorflow is pinning down to the patch release here (typing-extensions~=3.7.4)

Doesn't this mean any version at 3.7.4 or after but not after 3.8 or 4.0? That is, 3.7.5 could still be installed.

@matthewfeickert
Copy link

matthewfeickert commented Sep 7, 2021

Doesn't this mean any version at 3.7.4 or after but not after 3.8 or 4.0? That is, 3.7.5 could still be installed.

Yup, and indeed the compatible release syntax does that (get the latest in the v3.7.X series for X>=4 or more concisely "<3.8,>=3.7.4", which for typing-extensions is 3.7.4.3)

$ docker run --rm -ti python:3.9 /bin/bash
root@b4bbf0636f94:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@b4bbf0636f94:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@b4bbf0636f94:/# python -m pip --quiet install black tensorflow tensorflow-probability
(venv) root@b4bbf0636f94:/# python -m pip list | grep "pip\|black\|tensorflow\|typing"
black                   21.7b0
pip                     21.2.4
tensorflow              2.6.0
tensorflow-estimator    2.6.0
tensorflow-probability  0.13.0
typing-extensions       3.7.4.3

Python 3.8 struggles though

$ docker run --rm -ti python:3.8 /bin/bash
root@521504a6d9b2:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@521504a6d9b2:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@521504a6d9b2:/# python -m pip --quiet install black tensorflow tensorflow-probability
(venv) root@521504a6d9b2:/# python -m pip list | grep "pip\|black\|tensorflow\|typing"
black                   21.8b0
pip                     21.2.4
tensorflow              2.3.4
tensorflow-estimator    2.3.0
tensorflow-probability  0.13.0
typing-extensions       3.10.0.2
(venv) root@521504a6d9b2:/# python -c "import tensorflow as tf; import tensorflow_probability as tfp"
2021-09-07 22:21:56.457553: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-07 22:21:56.457576: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/__init__.py", line 20, in <module>
    from tensorflow_probability import substrates
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/substrates/__init__.py", line 21, in <module>
    from tensorflow_probability.python.internal import all_util
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py", line 142, in <module>
    dir(globals()[pkg_name])  # Forces loading the package from its lazy loader.
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 61, in __dir__
    module = self._load()
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 41, in _load
    self._on_first_access()
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py", line 63, in _validate_tf_environment
    raise ImportError(
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.5; Detected an installation of version 2.3.4. Please upgrade TensorFlow to proceed.

and same story for Python 3.7 (which isn't surprising given that it hits Python 3.8)

$ docker run --rm -ti python:3.7 /bin/bash
root@94133784a35f:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@94133784a35f:/# python --version --version
Python 3.7.11 (default, Sep  3 2021, 20:46:09) 
[GCC 10.2.1 20210110]
(venv) root@94133784a35f:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@94133784a35f:/# python -m pip --quiet install black tensorflow tensorflow-probability
(venv) root@94133784a35f:/# python -m pip list | grep "pip\|black\|tensorflow\|typing"
black                   21.8b0
pip                     21.2.4
tensorflow              2.3.4
tensorflow-estimator    2.3.0
tensorflow-probability  0.13.0
typing-extensions       3.10.0.2
(venv) root@94133784a35f:/# python -c "import tensorflow as tf; import tensorflow_probability as tfp"
2021-09-07 22:53:52.689934: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-07 22:53:52.690053: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/__init__.py", line 20, in <module>
    from tensorflow_probability import substrates
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/substrates/__init__.py", line 21, in <module>
    from tensorflow_probability.python.internal import all_util
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/python/__init__.py", line 142, in <module>
    dir(globals()[pkg_name])  # Forces loading the package from its lazy loader.
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 61, in __dir__
    module = self._load()
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 41, in _load
    self._on_first_access()
  File "/usr/local/venv/lib/python3.7/site-packages/tensorflow_probability/python/__init__.py", line 68, in _validate_tf_environment
    present=tf.__version__))
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.5; Detected an installation of version 2.3.4. Please upgrade TensorFlow to proceed.

@jvishnuvardhan jvishnuvardhan added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Sep 8, 2021
@bersbersbers
Copy link

bersbersbers commented Sep 15, 2021

This issue makes pip unable to install tensorflow==2.6.0 alongside astroid==2.8.0, compare
pylint-dev/astroid@40ea1a3#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52,
or black==21.8b0, see psf/black#2465.

@johnthagen
Copy link

johnthagen commented Sep 16, 2021

I had to temporarily pin an older version of black in my dev-requirements.in / dev-requirements.txt to work around this issue:

# TODO: Unpin black once upstream tensorflow conflicting dependency is resolved:
#    https://github.com/tensorflow/tensorflow/issues/51743
black == 21.7b0

@connorbrinton
Copy link

Hi @mihaimaruseac, I ran into this issue myself so I just submitted a very small PR that I believe should fix it (#52136). Let me know if there are any changes you would like to it! 🙂

@robertschweizer
Copy link

This seems to be fixed on master by 41e6614. The install_requires is now typing_extensions >= 3.6.6.

@mihaimaruseac
Copy link
Collaborator

Closing as this has been fixed while I was OOO.

@google-ml-butler
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

@bersbersbers
Copy link

bersbersbers commented Oct 8, 2021

For anyone wondering, this did not make it into tensorflow==2.7.0rc0.
https://github.com/tensorflow/tensorflow/blob/v2.7.0-rc0/tensorflow/tools/pip_package/setup.py#L94

@matthewfeickert
Copy link

For anyone wondering, this did not make it into tensorflow==2.7.0rc0.

This isn't too surprising to me as I would imagine that freezing the list of things that go into a release would happen several days if not week+ in advance. Releases are hard, especially for projects of this size.

This is good to know in advance though, so thank you for noting that!

@mihaimaruseac
Copy link
Collaborator

We need a cherry-pick of 41e6614 into r2.7 branch. We didn't do it while working on the RC0 release, but I think we should get it for the next RC.

@mihaimaruseac
Copy link
Collaborator

The commit has been cherry-picked, so the next release candidate should fully resolve this

@mihaimaruseac
Copy link
Collaborator

RC1 of TF 2.7 should already have this.

@woodyza
Copy link

woodyza commented Nov 29, 2021

Hi @mihaimaruseac, do you know if this is likely to be picked back into 2.6 at some point? I've just run into this there, and we hadn't planned on moving up to 2.7 just yet.

@mihaimaruseac
Copy link
Collaborator

Hi. Unfortunately we cannot do a cherry pick of this into TF 2.6 given the magnitude of the change and potential for breakages. Patch releases don't go through the same extensive QA process as final releases, so we're conservative in what changes get included there.

@woodyza
Copy link

woodyza commented Nov 30, 2021

Ok, yeah that full change is a lot more than I realised initially. Would a PR for just the specific loosening of the typing_extensions compatibility against 2.6 be something that would be entertained?

@mihaimaruseac
Copy link
Collaborator

That would potentially work

connorbrinton added a commit to connorbrinton/tensorflow that referenced this issue Nov 30, 2021
These changes loosen the version constraints on `typing-extensions` to
allow TensorFlow to be installed alongside packages that require
`typing-extensions >= 3.10`, such as newer versions of Black and PyLint.

Previously TensorFlow required a `typing-extensions` version in the
`3.7.x` series. However, the newer `3.10.x` series is fully
backwards-compatible with the `3.7.x` series (there are no `3.8.x` or
`3.9.x` series). Therefore, these changes allow `typing-extensions` to
be installed using either the `3.7.x` or `3.10.x` series.

In order to ensure that any future backwards-incompatible changes in the
`typing-extensions` package don't cause issues, TensorFlow's dependency
on `typing-extensions` does not allow versions from the `3.11.x` series
or beyond (`< 3.11`).

Fixes tensorflow#51743.
@connorbrinton
Copy link

@mihaimaruseac @woodyza Since there's consideration of including this change in a 2.6 release, I went ahead and rebased my changes on the r2.6 branch and opened a new PR here:
#53250

I would have just reopened the old PR, but I had deleted the fork, so that wasn't possible 😬

If there's a preference to more exactly match the changes cherry-picked into 2.7, I can change the version constraint to typing-extensions >= 3.6.6 🙂

@mihaimaruseac
Copy link
Collaborator

Awesome. Will review and approve and merge a while later when we do patch releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.6.0 stat:awaiting tensorflower Status - Awaiting response from tensorflower subtype:macOS macOS Build/Installation issues type:build/install Build and install issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.