Dependency resolver fails when version ranges are specified

[HotelCalifornia]

What did you want to do?

Attempting to install a package from an internal azure artifact feed (specified either as the only index url or as an extra index url), given a range of acceptable versions (e.g. >2.6.1, ~=3.0.0, ==3.*, etc) results in a failure to find a version satisfying the requirement according to pip. Despite, as seen in the output below, finding a number of seemingly qualifying candidates.

One thing to note is that all of the candidates for the version ranges I specified are considered release candidates per PEP 440, but testing e.g.:

>>> from packaging import version
>>> version.parse('3.1.5rc28') > version.parse('2.1.6')
True

seems to imply that the versions are valid and should satisfy the criteria.

A possibly related issue---pip was apparently unable to resolve the dependency click>=6,<7 (output) until specifying the version manually (output). (relevant requirements.txt)

Also note that the same issue occurs whether the azure artifact feed is specified as an index or not, i.e. pip install pros-cli>2.6.1 fails just the same.

Is pip just ignoring prereleases for the purposes of dependency resolution? Is there a way to force it to allow them?

`pip freeze`

artifacts-keyring==0.3.1
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
cryptography==3.4.6
idna==2.10
importlib-metadata==3.4.0
jeepney==0.6.0
keyring==22.0.1
pycparser==2.20
requests==2.25.1
SecretStorage==3.3.1
typing-extensions==3.7.4.3
urllib3==1.26.3
zipp==3.4.0

Output

the command output

$ pip install --verbose --no-cache-dir --extra-index-url https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/ 'pros-cli>=3.1.5'
Using pip 21.0.1 from /home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip (python 3.6)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-ubd_chgi
Created temporary directory: /tmp/pip-req-tracker-cy6nviwn
Initialized build tracking at /tmp/pip-req-tracker-cy6nviwn
Created build tracker: /tmp/pip-req-tracker-cy6nviwn
Entered build tracker: /tmp/pip-req-tracker-cy6nviwn
Created temporary directory: /tmp/pip-install-1hs0i6dy
Looking in indexes: https://pypi.org/simple, https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/
2 location(s) to search for versions of pros-cli:
* https://pypi.org/simple/pros-cli/
* https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/
Fetching project page and analyzing links: https://pypi.org/simple/pros-cli/
Getting page https://pypi.org/simple/pros-cli/
Found index url https://pypi.org/simple
Getting credentials from keyring for https://pypi.org/simple
Loading KWallet
Loading SecretService
Loading Windows
Loading chainer
Loading macOS
Loading ArtifactsKeyringBackend
Getting credentials from keyring for pypi.org
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/pros-cli/ HTTP/1.1" 200 1379
  Found link https://files.pythonhosted.org/packages/14/a7/61d58a91376e2bdc007100e1af003672f222a374e60a12454402353528e4/pros_cli-2.1.4-py3-none-any.whl#sha256=5ee8b9f798c9c4d5fa3ce24deed6aad0238201379672b335c6cec45e14ff5d74 (from https://pypi.org/simple/pros-cli/), version: 2.1.4
  Found link https://files.pythonhosted.org/packages/20/2e/a67478452de58c2a4aaac93f10b905f70863b962f5309b22e0a4eac69e3e/pros_cli-2.1.5-py3-none-any.whl#sha256=2bf46f4eed599e6b645693a71429758002e43810e2876517a61954e73a1c13c0 (from https://pypi.org/simple/pros-cli/), version: 2.1.5
  Found link https://files.pythonhosted.org/packages/36/95/fda454901979931a8562c9d3eb49865797a68b590b20ee2e650d8bf4b721/pros_cli-2.1.6-py3-none-any.whl#sha256=5c45265b10353037575ae1a1e4b04b3a5a1ec959158b986165973434605f39bb (from https://pypi.org/simple/pros-cli/), version: 2.1.6
  Found link https://files.pythonhosted.org/packages/e0/8e/556375ff92a03849e5b3ec0e65edb1a679f1bcf5d099c87023a38128daa8/pros_cli-2.1.7-py3-none-any.whl#sha256=b28f4571f9e395596d7d9716e89938ea3ee3006c563b1d8b03fadacd3cc6ddd0 (from https://pypi.org/simple/pros-cli/), version: 2.1.7
  Found link https://files.pythonhosted.org/packages/91/0c/6bb82388672a53b2d7afc90a0b4ccdbc0a26eb03b2a8ec06e45398a993e5/pros_cli-2.2.0-py3-none-any.whl#sha256=fe783facc89f0e9623dc4a39553b27211e91140761854b30f0f9afc0dd101770 (from https://pypi.org/simple/pros-cli/), version: 2.2.0
  Found link https://files.pythonhosted.org/packages/f0/cd/877d3b3ff11502042c1b2ae018cd80ea67eb712f73e44142da0d4d5afd58/pros_cli-2.4.3-py3-none-any.whl#sha256=dd9f1635503f728dd7f26ada441cc76cf7d525b0f7908ee613f953b6bea886c6 (from https://pypi.org/simple/pros-cli/), version: 2.4.3
  Found link https://files.pythonhosted.org/packages/da/64/87e6e6d7d848a06720199efa91fd57ca7b1a40ae7b60009bbd6e0953c1ec/pros_cli-2.5.0-py3-none-any.whl#sha256=db6a18c3faf7230cf5875ea2bca61267842e04cf99d3731163da2bf25f8ebc57 (from https://pypi.org/simple/pros-cli/), version: 2.5.0
  Found link https://files.pythonhosted.org/packages/db/5f/1fe5247988c1bc2d3eeb55348916fdceb5b8a6c708d4d5f9779c45a7f73d/pros_cli-2.5.1-py3-none-any.whl#sha256=9c0065053eaa3a078d0598ca12a741e050d3b5037ba83502d673fcc019619c8d (from https://pypi.org/simple/pros-cli/), version: 2.5.1
  Found link https://files.pythonhosted.org/packages/27/ea/09f3464906121888cd8e1f536c44b6d47391f7394e3568a6ae36a41a84c6/pros_cli-2.6.0-py3-none-any.whl#sha256=0126fa6b7ecd9d5b652d0e90115ff8c8ff2b551a5f310ff5452289d98dc1b240 (from https://pypi.org/simple/pros-cli/), version: 2.6.0
  Found link https://files.pythonhosted.org/packages/63/2b/49ad6d2179771cb9317c59a9709c749cb2872ac885890078d431fd41912b/pros_cli-2.6.1-py3-none-any.whl#sha256=ae3264ad878cc3be0a7e474ec11957f3b4d4704d2db04f206b57d892d4ea7131 (from https://pypi.org/simple/pros-cli/), version: 2.6.1
  Found link https://files.pythonhosted.org/packages/76/b2/6f682dcf6339ff67ae8a7f5ddee3c350a8e8b5318f2a0984bd2fca309f2c/pros_cli-3.1.5a18-py3-none-any.whl#sha256=25d991ac5f9ad18ff011efd9225848d963244f5fc84d86411abeb671230991a8 (from https://pypi.org/simple/pros-cli/), version: 3.1.5a18
  Found link https://files.pythonhosted.org/packages/ab/53/05012e3bb4c687387e038a4fc1af799901b8e110a2eb5a616e2348b00ffa/pros_cli-3.1.5rc33-py3-none-any.whl#sha256=77673232c04d3caae4e322f10924edea9141d00058c7a3529926a6e834cd0725 (from https://pypi.org/simple/pros-cli/), version: 3.1.5rc33
Fetching project page and analyzing links: https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/
Getting page https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/
Found index url https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/
Getting credentials from keyring for https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/
Starting new HTTPS connection (1): pkgs.dev.azure.com:443
https://pkgs.dev.azure.com:443 "GET /purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/ HTTP/1.1" 401 307
Starting new HTTPS connection (1): pkgs.dev.azure.com:443
https://pkgs.dev.azure.com:443 "GET /purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/ HTTP/1.1" 404 46
Found credentials in keyring for pkgs.dev.azure.com
Starting new HTTPS connection (1): pkgs.dev.azure.com:443
https://pkgs.dev.azure.com:443 "GET /purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/ HTTP/1.1" 200 None
  Found link https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/d0ae3b50-a8da-4378-a31b-580fbdf87f02/pypi/download/pros-cli/3.1.5rc28/pros_cli-3.1.5rc28-py3-none-any.whl#sha256=0414a9c41278b4536830a775ebdd119edaf4873ad49cb3d4b89c956eb358dd5e (from https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/), version: 3.1.5rc28
  Found link https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/d0ae3b50-a8da-4378-a31b-580fbdf87f02/pypi/download/pros-cli/3.1.5rc20/pros_cli-3.1.5rc20-py3-none-any.whl#sha256=66ce42469cfbf3c82882cac86fc9d8f4d634465e10678ff691280cbf6bcb8657 (from https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/), version: 3.1.5rc20
  Found link https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/d0ae3b50-a8da-4378-a31b-580fbdf87f02/pypi/download/pros-cli/3.1.5rc19/pros_cli-3.1.5rc19-py3-none-any.whl#sha256=3688efda5f62316aa93351a2e664c5d6fde5fdaffcd75747ea96fde3c8245716 (from https://pkgs.dev.azure.com/purdue-acm-sigbots/_packaging/pros-cli/pypi/simple/pros-cli/), version: 3.1.5rc19
Given no hashes to check 0 links for project 'pros-cli': discarding no candidates
ERROR: Could not find a version that satisfies the requirement pros-cli>=3.1.5
ERROR: No matching distribution found for pros-cli>=3.1.5
Exception information:
Traceback (most recent call last):
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    crit = self.state.criteria[name]
KeyError: 'pros-cli'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    name, crit = self._merge_into_criterion(r, parent=None)
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    crit = Criterion.from_requirement(self._p, requirement, parent)
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 83, in from_requirement
    raise RequirementsConflicted(criterion)
pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('pros-cli>=3.1.5')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 320, in resolve
    raise ResolutionImpossible(e.criterion.information)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('pros-cli>=3.1.5'), parent=None)]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 189, in _main
    status = self.run(options, args)
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 317, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/home/hotel/.pyenv/versions/3.6.12/envs/azure-test/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 127, in resolve
    six.raise_from(error, e)
  File "<string>", line 3, in raise_from
pip._internal.exceptions.DistributionNotFound: No matching distribution found for pros-cli>=3.1.5

[uranusjr]

I believe this is the same as #9197 (already fixed, will be in the next release), but will need to double check.

[HotelCalifornia]

thanks for the response!

[HotelCalifornia]

that said, do #9613 and #9197 include specifiers other than ~=? because i'm getting this with anything that isn't ==

[uranusjr]

#9197 is about the rc part of the specifier was split incorrectly when comparing; ~= behaving incorrectly is one sympton to the cause. But I’m not entirely sure and will need to check.

[pradyunsg]

Closing since this is likely fixed.