Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot install tensorflow and opencv-python at the same time due to wrongly detected numpy requirement #4451

Closed
3 tasks done
RafalSkolasinski opened this issue Aug 29, 2021 · 3 comments
Closed
3 tasks done

cannot install tensorflow and opencv-python at the same time due to wrongly detected numpy requirement #4451

RafalSkolasinski opened this issue Aug 29, 2021 · 3 comments
Labels
kind/bug Something isn't working as expected

Comments

@RafalSkolasinski
Copy link

RafalSkolasinski commented Aug 29, 2021 •
edited

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: Ubuntu 20.10
  • Poetry version: 1.1.8
  • Link of a Gist with the contents of your pyproject.toml file:

Issue

I am trying to create pyproject.toml that would specify both tensorflow and opencv-python as dependencies.
This lead to `SolverProblemError

  Because no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
   and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
  And because tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
  Because no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
   and opencv-python (4.5.3.56) depends on numpy (>=1.21.0), opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0).
  Thus, opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0).
  So, because poetry-opencv-tensorflow depends on both tensorflow (~2.5.0) and opencv-python (^4.5.3), version solving failed.

So, as tensorflow has very strict numpy requirement this must be 1.19. Assuming here this would python 3.8 the opencv would require minimum numpy 1.17.3 (so 1.19 would be fine).
For some reason poetry thinks it requires 1.21 which is incorrect according to:

  1. setup.py of opencv-python, see github
  2. metadata of downloaded wheel
$ pkginfo -f requires_dist opencv_python-4.5.3.56-cp38-cp38-win_amd64.whl
requires_dist: ['numpy (>=1.17.3)']
  1. fact that it works fine with pip
$ pip freeze | grep -P "numpy|tensorflow"                  
numpy==1.19.5
tensorflow==2.5.0
tensorflow-estimator==2.5.0

$ pip install opencv-python       
Collecting opencv-python
  Using cached opencv_python-4.5.3.56-cp38-cp38-manylinux2014_x86_64.whl (49.9 MB)
Requirement already satisfied: numpy>=1.17.3 in /home/rskolasinski/miniconda3/envs/test/lib/python3.8/site-packages (from opencv-python) (1.19.5)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.5.3.56

To reproduce:

First prepare environment from linked gist file

$ poetry install
Creating virtualenv poetry-opencv-tensorflow-qJ4q1krR-py3.8 in /home/rskolasinski/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (2.1s)

Writing lock file

Package operations: 36 installs, 0 updates, 0 removals

  • Installing certifi (2021.5.30)
  • Installing charset-normalizer (2.0.4)
  • Installing idna (3.2)
  • Installing pyasn1 (0.4.8)
  • Installing urllib3 (1.26.6)
  • Installing cachetools (4.2.2)
  • Installing oauthlib (3.1.1)
  • Installing pyasn1-modules (0.2.8)
  • Installing requests (2.26.0)
  • Installing rsa (4.7.2)
  • Installing six (1.15.0)
  • Installing google-auth (1.35.0)
  • Installing requests-oauthlib (1.3.0)
  • Installing absl-py (0.13.0)
  • Installing grpcio (1.34.1)
  • Installing google-auth-oauthlib (0.4.5)
  • Installing markdown (3.3.4)
  • Installing numpy (1.19.5)
  • Installing protobuf (3.17.3)
  • Installing tensorboard-plugin-wit (1.8.0)
  • Installing tensorboard-data-server (0.6.1)
  • Installing werkzeug (2.0.1)
  • Installing astunparse (1.6.3)
  • Installing flatbuffers (1.12)
  • Installing gast (0.4.0)
  • Installing google-pasta (0.2.0)
  • Installing h5py (3.1.0)
  • Installing keras-nightly (2.5.0.dev2021032900)
  • Installing keras-preprocessing (1.1.2)
  • Installing opt-einsum (3.3.0)
  • Installing tensorboard (2.6.0)
  • Installing tensorflow-estimator (2.5.0)
  • Installing termcolor (1.1.0)
  • Installing typing-extensions (3.7.4.3)
  • Installing wrapt (1.12.1)
  • Installing tensorflow (2.5.1)

and now try to add opencv-python package

$ poetry add opencv-python -vvv
Using virtualenv: /home/rskolasinski/.cache/pypoetry/virtualenvs/poetry-opencv-tensorflow-qJ4q1krR-py3.8
PyPI: No release information found for opencv-python-3.1.0, skipping
PyPI: 53 packages found for opencv-python *
Using version ^4.5.3 for opencv-python

Updating dependencies
Resolving dependencies...
   1: fact: poetry-opencv-tensorflow is 0.1.0
   1: derived: poetry-opencv-tensorflow
   1: fact: poetry-opencv-tensorflow depends on tensorflow (~2.5.0)
   1: fact: poetry-opencv-tensorflow depends on opencv-python (^4.5.3)
   1: selecting poetry-opencv-tensorflow (0.1.0)
   1: derived: opencv-python (>=4.5.3,<5.0.0)
   1: derived: tensorflow (>=2.5.0,<2.6.0)
PyPI: No release information found for opencv-python-3.1.0, skipping
PyPI: 1 packages found for opencv-python >=4.5.3,<5.0.0
   1: fact: opencv-python (4.5.3.56) depends on numpy (>=1.21.0)
   1: selecting opencv-python (4.5.3.56)
   1: derived: numpy (>=1.21.0)
PyPI: No release information found for numpy-0.9.6, skipping
PyPI: No release information found for numpy-0.9.8, skipping
PyPI: No release information found for numpy-1.0.3, skipping
PyPI: No release information found for numpy-1.0.4, skipping
PyPI: No release information found for numpy-1.0b1, skipping
PyPI: No release information found for numpy-1.0b4, skipping
PyPI: No release information found for numpy-1.0b5, skipping
PyPI: No release information found for numpy-1.0rc1, skipping
PyPI: No release information found for numpy-1.0rc2, skipping
PyPI: No release information found for numpy-1.0rc3, skipping
PyPI: No release information found for numpy-1.1.1, skipping
PyPI: No release information found for numpy-1.2.0, skipping
PyPI: No release information found for numpy-1.2.1, skipping
PyPI: No release information found for numpy-1.4.0, skipping
PyPI: 3 packages found for numpy >=1.21.0
   1: fact: tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0)
   1: fact: tensorflow (2.5.1) depends on absl-py (>=0.10,<1.0)
   1: fact: tensorflow (2.5.1) depends on astunparse (>=1.6.3,<1.7.0)
   1: fact: tensorflow (2.5.1) depends on flatbuffers (>=1.12.0,<1.13.0)
   1: fact: tensorflow (2.5.1) depends on google-pasta (>=0.2,<1.0)
   1: fact: tensorflow (2.5.1) depends on h5py (>=3.1.0,<3.2.0)
   1: fact: tensorflow (2.5.1) depends on keras-preprocessing (>=1.1.2,<1.2.0)
   1: fact: tensorflow (2.5.1) depends on opt-einsum (>=3.3.0,<3.4.0)
   1: fact: tensorflow (2.5.1) depends on protobuf (>=3.9.2)
   1: fact: tensorflow (2.5.1) depends on six (>=1.15.0,<1.16.0)
   1: fact: tensorflow (2.5.1) depends on termcolor (>=1.1.0,<1.2.0)
   1: fact: tensorflow (2.5.1) depends on typing-extensions (>=3.7.4,<3.8.0)
   1: fact: tensorflow (2.5.1) depends on wrapt (>=1.12.1,<1.13.0)
   1: fact: tensorflow (2.5.1) depends on gast (0.4.0)
   1: fact: tensorflow (2.5.1) depends on tensorboard (>=2.5,<3.0)
   1: fact: tensorflow (2.5.1) depends on tensorflow-estimator (>=2.5.0,<2.6.0)
   1: fact: tensorflow (2.5.1) depends on keras-nightly (>=2.5.0.dev,<2.6.0)
   1: fact: tensorflow (2.5.1) depends on grpcio (>=1.34.0,<1.35.0)
   1: derived: not tensorflow (==2.5.1)
PyPI: 1 packages found for tensorflow >=2.5.0,<2.5.1 || >2.5.1,<2.6.0
   1: fact: tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0)
   1: fact: tensorflow (2.5.0) depends on absl-py (>=0.10,<1.0)
   1: fact: tensorflow (2.5.0) depends on astunparse (>=1.6.3,<1.7.0)
   1: fact: tensorflow (2.5.0) depends on flatbuffers (>=1.12.0,<1.13.0)
   1: fact: tensorflow (2.5.0) depends on google-pasta (>=0.2,<1.0)
   1: fact: tensorflow (2.5.0) depends on h5py (>=3.1.0,<3.2.0)
   1: fact: tensorflow (2.5.0) depends on keras-preprocessing (>=1.1.2,<1.2.0)
   1: fact: tensorflow (2.5.0) depends on opt-einsum (>=3.3.0,<3.4.0)
   1: fact: tensorflow (2.5.0) depends on protobuf (>=3.9.2)
   1: fact: tensorflow (2.5.0) depends on six (>=1.15.0,<1.16.0)
   1: fact: tensorflow (2.5.0) depends on termcolor (>=1.1.0,<1.2.0)
   1: fact: tensorflow (2.5.0) depends on typing-extensions (>=3.7.4,<3.8.0)
   1: fact: tensorflow (2.5.0) depends on wrapt (>=1.12.1,<1.13.0)
   1: fact: tensorflow (2.5.0) depends on gast (0.4.0)
   1: fact: tensorflow (2.5.0) depends on tensorboard (>=2.5,<3.0)
   1: fact: tensorflow (2.5.0) depends on tensorflow-estimator (>=2.5.0rc0,<2.6.0)
   1: fact: tensorflow (2.5.0) depends on keras-nightly (>=2.5.0.dev,<2.6.0)
   1: fact: tensorflow (2.5.0) depends on grpcio (>=1.34.0,<1.35.0)
   1: derived: not tensorflow (==2.5.0)
   1: fact: no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
   1: conflict: no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
   1: !  tensorflow (>2.5.0,<2.5.1 || >2.5.1,<2.6.0) is partially satisfied by not  tensorflow (2.5.0)
   1: ! which is caused by "tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0)"
   1: ! thus: tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
   1: !  tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) is partially satisfied by not  tensorflow (2.5.1)
   1: ! which is caused by "tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0)"
   1: ! thus: tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
   1: fact: tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
   1: derived: numpy (>=1.19.2,<1.20.0)
   1: derived: not opencv-python (==4.5.3.56)
   1: fact: no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
   1: conflict: no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
   1: !  opencv-python (>=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0) is partially satisfied by not  opencv-python (4.5.3.56)
   1: ! which is caused by "opencv-python (4.5.3.56) depends on numpy (>=1.21.0)"
   1: ! thus: opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0)
   1: ! not  numpy (>=1.21.0) is satisfied by  numpy (>=1.19.2,<1.20.0)
   1: ! which is caused by "tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)"
   1: ! thus: opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0)
   1: !  tensorflow (>=2.5.0,<2.6.0) is satisfied by  tensorflow (~2.5.0)
   1: ! which is caused by "poetry-opencv-tensorflow depends on tensorflow (~2.5.0)"
   1: ! thus: opencv-python is forbidden
   1: !  opencv-python (>=4.5.3,<5.0.0) is satisfied by  opencv-python (^4.5.3)
   1: ! which is caused by "poetry-opencv-tensorflow depends on opencv-python (^4.5.3)"
   1: ! thus: version solving failed
   1: Version solving took 0.241 seconds.
   1: Tried 1 solutions.

  Stack trace:

  8  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  7  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  6  ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  5  ~/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  4  ~/.poetry/lib/poetry/console/commands/add.py:173 in handle
      171│             self._installer.whitelist([r["name"] for r in requirements])
      172│ 
    → 173│             status = self._installer.run()
      174│         except BaseException:
      175│             # Using BaseException here as some exceptions, eg: KeyboardInterrupt, do not inherit from Exception

  3  ~/.poetry/lib/poetry/installation/installer.py:103 in run
      101│         local_repo = Repository()
      102│ 
    → 103│         return self._do_install(local_repo)
      104│ 
      105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  2  ~/.poetry/lib/poetry/installation/installer.py:235 in _do_install
      233│             )
      234│ 
    → 235│             ops = solver.solve(use_latest=self._whitelist)
      236│         else:
      237│             self._io.write_line("Installing dependencies from lock file")

  1  ~/.poetry/lib/poetry/puzzle/solver.py:65 in solve
       63│         with self._provider.progress():
       64│             start = time.time()
    →  65│             packages, depths = self._solve(use_latest=use_latest)
       66│             end = time.time()
       67│ 

  SolverProblemError

  Because no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
   and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
  And because tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
  Because no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
   and opencv-python (4.5.3.56) depends on numpy (>=1.21.0), opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0).
  Thus, opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0).
  So, because poetry-opencv-tensorflow depends on both tensorflow (~2.5.0) and opencv-python (^4.5.3), version solving failed.

  at ~/.poetry/lib/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes
@RafalSkolasinski RafalSkolasinski added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Aug 29, 2021
@RafalSkolasinski
Copy link
Author

@finswimmer

@dimbleby
Copy link
Contributor

For some reason poetry thinks it requires 1.21 which is incorrect ...

because that's what the JSON API reports:

$ curl -s https://pypi.org/pypi/opencv-python/4.5.3.56/json | jq '.info.requires_dist'
[
  "numpy (>=1.21.0)"
]

ask opencv-python to publish consistent requirements, if this is still a problem for you.

Not a poetry bug.

@p-wojciechowski p-wojciechowski mentioned this issue Sep 8, 2022
Closed
3 tasks
@mkniewallner mkniewallner removed the status/triage This issue needs to be triaged label Sep 18, 2022
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 1, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
kind/bug Something isn't working as expected
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dimbleby @neersighted @mkniewallner @RafalSkolasinski