Pip fails on different compatible ~= version specifiers

[coreydexter]

Description

Say we have four wheels

• package_1-1.0.0: no dependencies
• package_1-1.1.0: no dependencies
• package_2-0.0.1: depends on package_1~=1.0.0
• package_3-0.0.1: depends on package_1~=1.0 and package_2

When trying to install package_3 pip fails with

pip._vendor.resolvelib.resolvers.InconsistentCandidate: Provided candidate LinkCandidate('file:///home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.1.0-py3-none-any.whl') does not satisfy SpecifierRequirement('package-1~=1.0'), SpecifierRequirement('package-1~=1.0.0')

Expected behavior

pip should install a version of package_1 that satisfies both of the package_2 and package_3 requirements. That is it should install package_1-1.0.0.

pip version

23.2.1

Python version

3.9.17

OS

Ubuntu 20.04.4 LTS

How to Reproduce

#!/bin/bash
$ python3.9 -m venv env_reproduce
$ source env_reproduce/bin/activate

$ pip install pip==23.2.1

$ mkdir package_1
$ mkdir package_2
$ mkdir package_3

$ echo '[project]' >> package_1/pyproject.toml
$ echo 'name = "package_1"' >> package_1/pyproject.toml
$ echo 'version = "1.0.0"' >> package_1/pyproject.toml
$ echo 'dependencies = [] ' >> package_1/pyproject.toml

$ echo '[project]' >> package_2/pyproject.toml
$ echo 'name = "package_2"' >> package_2/pyproject.toml
$ echo 'version = "0.0.1"' >> package_2/pyproject.toml
$ echo 'dependencies = ["package_1~=1.0.0"] ' >> package_2/pyproject.toml

$ echo '[project]' >> package_3/pyproject.toml
$ echo 'name = "package_3"' >> package_3/pyproject.toml
$ echo 'version = "0.0.1"' >> package_3/pyproject.toml
$ echo 'dependencies = ["package_1~=1.0", "package_2"] ' >> package_3/pyproject.toml

$ cd package_1
$ pip wheel . -w ../dist
$ sed -i 's/1.0.0/1.1.0/g' pyproject.toml
$ pip wheel . -w ../dist

$ cd ../package_2
$ pip wheel . -w ../dist --find-links ../dist

$ cd ../package_3
$ pip wheel . -w ../dist --find-links ../dist

Output

$ python3.9 -m venv env_reproduce
$ source env_reproduce/bin/activate
$ pip install pip==23.2.1
Looking in indexes: 
Collecting pip==23.2.1
  Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-23.2.1

$ mkdir package_1
$ mkdir package_2
$ mkdir package_3

$ echo '[project]'
$ echo 'name = "package_1"'
$ echo 'version = "1.0.0"'
$ echo 'dependencies = [] '

$ echo '[project]'
$ echo 'name = "package_2"'
$ echo 'version = "0.0.1"'
$ echo 'dependencies = ["package_1~=1.0.0"] '

$ echo '[project]'
$ echo 'name = "package_3"'
$ echo 'version = "0.0.1"'
$ echo 'dependencies = ["package_1~=1.0", "package_2"] '

$ cd package_1
$ pip wheel . -w ../dist
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/package_1
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: package-1
  Building wheel for package-1 (pyproject.toml) ... done
  Created wheel for package-1: filename=package_1-1.0.0-py3-none-any.whl size=947 sha256=0221d34364bc0b17a1e4fc6f9643efda42b0bbc6ebfcae22599e09215a9e08b9
  Stored in directory: /tmp/pip-ephem-wheel-cache-cqd0il7j/wheels/32/40/3a/74a8426df9d035840cc61d8177eb882251907a6b7f7803d335
Successfully built package-1
$ sed -i s/1.0.0/1.1.0/g pyproject.toml
$ pip wheel . -w ../dist
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/package_1
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: package-1
  Building wheel for package-1 (pyproject.toml) ... done
  Created wheel for package-1: filename=package_1-1.1.0-py3-none-any.whl size=947 sha256=4f3fc75f08656839abb41b3b3addb922ef69cf994f0c8a5af3cc001c441c938b
  Stored in directory: /tmp/pip-ephem-wheel-cache-93wa19lf/wheels/32/40/3a/74a8426df9d035840cc61d8177eb882251907a6b7f7803d335
Successfully built package-1

$ cd ../package_2
$ pip wheel . -w ../dist --find-links ../dist
Looking in links: ../dist
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/package_2
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.0.0-py3-none-any.whl (from package-2==0.0.1)
  File was already downloaded /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.0.0-py3-none-any.whl
Building wheels for collected packages: package-2
  Building wheel for package-2 (pyproject.toml) ... done
  Created wheel for package-2: filename=package_2-0.0.1-py3-none-any.whl size=969 sha256=50e9803422d717cb04d1a0385880220c4bc8d4e74af129475fe34250d8761856
  Stored in directory: /tmp/pip-ephem-wheel-cache-wp0cfqi7/wheels/c9/09/93/be71d67a8b4efcbed4e78d6c247c33802b9064a0d3967a7a13
Successfully built package-2

$ cd ../package_3
$ pip wheel . -w ../dist --find-links ../dist
Looking in links: ../dist
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/package_3
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.1.0-py3-none-any.whl (from package-3==0.0.1)
  File was already downloaded /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.1.0-py3-none-any.whl
Processing /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_2-0.0.1-py3-none-any.whl (from package-3==0.0.1)
  File was already downloaded /home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_2-0.0.1-py3-none-any.whl
ERROR: Exception:
Traceback (most recent call last):
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_internal/commands/wheel.py", line 147, in run
    requirement_set = resolver.resolve(reqs, check_supported_wheels=True)
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/home/corey/repos/tmp/pip_inconsistent_candidate/testing/env_reproduce/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 254, in _attempt_to_pin_criterion
    raise InconsistentCandidate(candidate, criterion)
pip._vendor.resolvelib.resolvers.InconsistentCandidate: Provided candidate LinkCandidate('file:///home/corey/repos/tmp/pip_inconsistent_candidate/testing/dist/package_1-1.1.0-py3-none-any.whl') does not satisfy SpecifierRequirement('package-1~=1.0'), SpecifierRequirement('package-1~=1.0.0')

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

~=1.0.0 only allows 1.0.x and not 1.1, according to PEP 440. This is correct.

https://peps.python.org/pep-0440/#compatible-release

[uranusjr]

Sorry I misread. pip should find 1.0.0 as mentioned.

[uranusjr]

From a first look this seems to be a bug in the provider. I think the provider is incorrectly supplying 1.1.0 as a compatible version, but the resolver correctly identifies this as an inconsistency.

[coreydexter]

I had a poke around it looks like the issue is coming from the SpecifierSet, in particular the eq function of Specifier. This results in set operations thinking the two versions are equal, and the result can change depending on the ordering of the sets.

As an example

>>> from pip._vendor.packaging.specifiers import Specifier, SpecifierSet
>>> Specifier("~=1.0") == Specifier("~=1.0.0")
True
>>> set([Specifier("~=1.0")]) | set([Specifier("~=1.0.0")])
{<Specifier('~=1.0')>}
>>> set([Specifier("~=1.0.0")]) | set([Specifier("~=1.0")]) 
{<Specifier('~=1.0.0')>}

This appears to be because the canonicalize_version function strips off the trailing .0 to normalise the version, however that is problematic in this case.

And so when the provider tries to combine the specifiers, it's resulting in only the ~=1.0 one being kept.

[coreydexter]

I've confirmed that this issue is fixed by the packaging fix pypa/packaging#493.

So I will close this issue as a duplicate of #9613.