[BUG] Unable to install prospector with pipenv witout additional dependency

[sbrunner]

Describe the bug
pypenv install prospector==1.4.0 returns the following error

Installing prospector==1.4.0...
Adding prospector to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed! 
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/resolver.py", line 741, in _main
[ResolutionFailure]:       resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/resolver.py", line 702, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/resolver.py", line 684, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/utils.py", line 1397, in resolve_deps
[ResolutionFailure]:       results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/utils.py", line 1110, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pipenv/utils.py", line 835, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches pycodestyle<2.7.0,<2.8.0,>=2.6.0,>=2.7.0 (from prospector==1.4.0->-r /tmp/pipenvbpqmk53mrequirements/pipenv-pvtoxxy9-constraints.txt (line 2))
Tried: 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.3.1, 2.3.1, 2.4.0, 2.4.0, 2.5.0, 2.5.0, 2.6.0, 2.6.0, 2.7.0, 2.7.0
Skipped pre-versions: 1.8.0.dev0, 1.8.0.dev0, 2.0.0a1, 2.0.0a1, 2.6.0a1, 2.6.0a1
There are incompatible versions in the resolved dependencies:
  pycodestyle<2.7.0,>=2.6.0 (from prospector==1.4.0->-r /tmp/pipenvbpqmk53mrequirements/pipenv-pvtoxxy9-constraints.txt (line 2))
  pycodestyle<2.8.0,>=2.7.0 (from flake8==3.9.2->flake8-polyfill==1.0.2->pep8-naming==0.10.0->prospector==1.4.0->-r /tmp/pipenvbpqmk53mrequirements/pipenv-pvtoxxy9-constraints.txt (line 2))

A workaround is to run: pipenv install prospector==1.4.0 flake8==3.8.4

The solution is probably to update pyflake...

To Reproduce
Steps to reproduce the behavior:

1. Run pypenv install prospector==1.4.0 from a clean folder

Expected behavior
Install prospector without any error

Environment (please complete the following information):

• OS: Linux
• Prospector 1.4.0
• Python 3.8.6
• pipenv, version 2021.5.29
• pip: 21.1.3

[Pierre-Sassoulas]

I think this was fixed in #414 , could you try to install the latest master, please ? We'll release 1.4.1 shortly.

[sbrunner]

I confirm that's fixed, Thanks :-)

[Kilo59]

I think the issue might be back with the 1.5 release

ERROR: Could not find a version that matches pyflakes<2.3.0,<2.4.0,>=2.2.0,>=2.3.0 (from prospector==1.5.0->-r /var/folders/d8/tqjtc5rd1sngz3fjgs2xgnb00000gp/T/pipenvy3tl3i4frequirements/pipenv-u8uml7zi-constraints.txt (line 2))
Tried: 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.6.1, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.8.1, 0.8.1, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.3.1, 2.3.1
There are incompatible versions in the resolved dependencies:
  pyflakes<2.3.0,>=2.2.0 (from prospector==1.5.0->-r /var/folders/d8/tqjtc5rd1sngz3fjgs2xgnb00000gp/T/pipenvy3tl3i4frequirements/pipenv-u8uml7zi-constraints.txt (line 2))
  pyflakes<2.4.0,>=2.3.0 (from flake8==3.9.2->flake8-polyfill==1.0.2->pep8-naming==0.10.0->prospector==1.5.0->-r /var/folders/d8/tqjtc5rd1sngz3fjgs2xgnb00000gp/T/pipenvy3tl3i4frequirements/pipenv-u8uml7zi-constraints.txt (line 2))

This is for Python 3.9 on Mac.

Works with flake8="<3.9"

pipenv 2020.11.15
pip 21.0.1  

[carlio]

What version of pip are you using? I had tried installing this myself locally with pip from the PyPI version, and also the poetry dependency resolver had no issue.

Also, could you try installing the latest develop branch to see if that fixes it?

[sbrunner]

I do a dirty test, but why not upgrade pyflakes?
master...sbrunner:pyflakes

[sbrunner]

Also, could you try installing the latest develop branch to see if that fixes it?

The dependency definitions are specific to poetry, then it's difficult to test it with pipenv, do you have a quick way to do that?

[agalazis]

The issue that they lock to minor releases, while they are not supposed to break things . If their maintainers are sensible it is sufficient to lock to major release

[okuuva]

pip-tools has the same issue (IIRC pipenv uses pip-tools under the hood):

System info:
macOS Catalina 10.15.7 (19H1323)

❯ python3 --version
Python 3.9.4

❯ pip freeze
click==8.0.1
pep517==0.11.0
pip-tools==6.2.0
tomli==1.2.1

❯ cat requirements.in
prospector[with_everything]==1.5.0.1

❯ pip-compile requirements.in
Could not find a version that matches pyflakes<2.3.0,<2.4.0,>=2.2.0,>=2.3.0 (from prospector[with_everything]==1.5.0.1->-r requirements.in (line 1))
Tried: 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.6.1, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.8.1, 0.8.1, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.3.1, 2.3.1
There are incompatible versions in the resolved dependencies:
  pyflakes<2.4.0,>=2.3.0 (from flake8==3.9.2->flake8-polyfill==1.0.2->pep8-naming==0.10.0->prospector[with_everything]==1.5.0.1->-r requirements.in (line 1))
  pyflakes<2.3.0,>=2.2.0 (from prospector[with_everything]==1.5.0.1->-r requirements.in (line 1))

I'm trying to figure out if there's a way to provide a locally built tar/wheel in requirements.in to see if a package built from develop branch fixes the issue.

Edit: Figured it out, did not help:

❯ python3 --version
Python 3.9.4

❯ pip freeze
backports.entry-points-selectable==1.1.0
CacheControl==0.12.6
cachy==0.3.0
certifi==2021.5.30
charset-normalizer==2.0.4
cleo==0.8.1
clikit==0.6.2
crashtest==0.3.1
distlib==0.3.2
filelock==3.0.12
html5lib==1.1
idna==3.2
keyring==21.8.0
lockfile==0.12.2
msgpack==1.0.2
packaging==20.9
pastel==0.2.1
pexpect==4.8.0
pkginfo==1.7.1
platformdirs==2.3.0
poetry==1.1.8
poetry-core==1.0.4
ptyprocess==0.7.0
pylev==1.4.0
pyparsing==2.4.7
requests==2.26.0
requests-toolbelt==0.9.1
shellingham==1.4.0
six==1.16.0
tomlkit==0.7.2
urllib3==1.26.6
virtualenv==20.7.2
webencodings==0.5.1

# (Boy, Poetry sure does have a lot of dependencies...)

MacBook-Pro in prospector on  develop (75717e8) is 📦 v1.5.0.1 via 🐍 v3.9.4 (prospector)
❯ poetry build
Building prospector (1.5.0.1)
  - Building sdist
  - Built prospector-1.5.0.1.tar.gz
  - Building wheel
  - Built prospector-1.5.0.1-py3-none-any.whl
 
❯ pip freeze
click==8.0.1
pep517==0.11.0
pip-tools==6.2.0
tomli==1.2.1

❯ cat requirements.in
./prospector-1.5.0.1-py3-none-any.whl

❯ pip-compile requirements.in
Could not find a version that matches pyflakes<2.3.0,<2.4.0,>=2.2.0,>=2.3.0 (from flake8==3.9.2->flake8-polyfill==1.0.2->pep8-naming==0.10.0->prospector==1.5.0.1->-r requirements.in (line 1))
Tried: 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.6.1, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.8.1, 0.8.1, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 1.0.0, 1.0.0, 1.1.0, 1.1.0, 1.2.0, 1.2.0, 1.2.1, 1.2.1, 1.2.2, 1.2.2, 1.2.3, 1.2.3, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 2.1.1, 2.1.1, 2.2.0, 2.2.0, 2.3.0, 2.3.0, 2.3.1, 2.3.1
There are incompatible versions in the resolved dependencies:
  pyflakes<2.4.0,>=2.3.0 (from flake8==3.9.2->flake8-polyfill==1.0.2->pep8-naming==0.10.0->prospector==1.5.0.1->-r requirements.in (line 1))
  pyflakes<2.3.0,>=2.2.0 (from prospector==1.5.0.1->-r requirements.in (line 1))

[okuuva]

Interestingly enough poetry doesn't barf but boy does it have to dig deep in order to find a combination of dependencies that works:

❯ python3 --version
Python 3.9.4

❯ pip freeze
backports.entry-points-selectable==1.1.0
CacheControl==0.12.6
cachy==0.3.0
certifi==2021.5.30
charset-normalizer==2.0.4
cleo==0.8.1
clikit==0.6.2
crashtest==0.3.1
distlib==0.3.2
filelock==3.0.12
html5lib==1.1
idna==3.2
keyring==21.8.0
lockfile==0.12.2
msgpack==1.0.2
packaging==20.9
pastel==0.2.1
pexpect==4.8.0
pkginfo==1.7.1
platformdirs==2.3.0
poetry==1.1.8
poetry-core==1.0.4
ptyprocess==0.7.0
pylev==1.4.0
pyparsing==2.4.7
requests==2.26.0
requests-toolbelt==0.9.1
shellingham==1.4.0
six==1.16.0
tomlkit==0.7.2
urllib3==1.26.6
virtualenv==20.7.2
webencodings==0.5.1

MacBook-Pro in ~/prospector is 📦 v0.1.0 via 🐍 v3.9.4 (prospector)
❯ cat pyproject.toml
[tool.poetry]
name = "prospector-dependency-test"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.9"
prospector = {extras = ["with_everything"], version = "^1.5.0"}

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

❯ poetry install
Updating dependencies
Resolving dependencies... (44.5s)

Writing lock file

Package operations: 39 installs, 0 updates, 0 removals

  • Installing lazy-object-proxy (1.6.0)
  • Installing wrapt (1.12.1)
  • Installing astroid (2.7.3)
  • Installing isort (5.9.3)
  • Installing mccabe (0.6.1)
  • Installing pep8 (1.7.1)
  • Installing pyflakes (2.2.0)
  • Installing smmap (4.0.0)
  • Installing toml (0.10.2)
  • Installing flake8 (2.3.0)
  • Installing gitdb (4.0.7)
  • Installing pbr (5.6.0)
  • Installing pylint (2.10.2)
  • Installing typing-extensions (3.10.0.2)
  • Installing docutils (0.17.1)
  • Installing flake8-polyfill (1.0.2)
  • Installing gitpython (3.1.20)
  • Installing mypy-extensions (0.4.3)
  • Installing pies (2.6.7)
  • Installing pygments (2.10.0)
  • Installing pylint-plugin-utils (0.6)
  • Installing pyyaml (5.4.1)
  • Installing snowballstemmer (2.1.0)
  • Installing stevedore (3.4.0)
  • Installing bandit (1.7.0)
  • Installing dodgy (0.2.1)
  • Installing frosted (1.4.1)
  • Installing mypy (0.910)
  • Installing pep8-naming (0.10.0)
  • Installing pycodestyle (2.7.0)
  • Installing pydocstyle (6.1.1)
  • Installing pylint-celery (0.3)
  • Installing pylint-django (2.4.4)
  • Installing pylint-flask (0.6)
  • Installing pyroma (3.2)
  • Installing requirements-detector (0.7)
  • Installing setoptconf (0.2.0)
  • Installing vulture (2.3)
  • Installing prospector (1.5.0.1)

flake8 goes all the way back to 2.3.0 released in Jan 4, 2015 (!)

[okuuva]

Ok, this might be actually a bug in pip-tools. Here's the dependency chain:
prospector -> pep8-naming = ">=0.3.3,<=0.10.0" (resolves to pep8-naming==0.10.0) -> flake8_polyfill>=1.0.2,<2 (resolves to flake8_polyfill==1.0.2) -> flake8 (no version range provided, resolves to the newest one)

Bumping pyflakes version range to pyflakes<2.4.0,>=2.3.0 would be a workaround (then you could also bump pep8-naming to pep8-naming==0.12.1 which requires flake8>=3.9.1. Other would be binding the version range for flake8 yourselves (frankly I was really surprised you don't do that already).

[Pierre-Sassoulas]

I would merge a pull request that introduce range for all requirements instead of fixed version. (We already handled pylint and pyflakes but doing them one by one is not very efficient).