Dependency resolution failure with git packages from different sources (editable mode)

[neradha]

Description

Installations from git repositories treat different sources as different versions even when they are the same version. For example, using pip install -e . to install a repository in editable mode whose dependencies include this project itself but on github always fail because the local version is considered different to the github version, even when they are the same version.

Expected behavior

Pip should treat packages with the same name and same version as being identical, even when they were installed from different sources (particularly relevant here for installation from git repositories).

pip version

21.1.1

Python version

3.8.5

OS

Windows 10

How to Reproduce

Working:

1. Run pip install git+https://github.com/neradha/my-package.git

This works because my-package is coming from github in both cases.

Failing:

1. Clone https://github.com/neradha/my-package.git locally
2. Run pip install -e . in that directory

Pip installs the package from the local source, it follows the dependencies to install https://github.com/neradha/my-other-package which itself relies on my-package. This dependency specifies github which causes the resolver to fail.

Output

Obtaining file:///C:/path_on_my_pc/my-package
Collecting my-other-package@ git+https://github.com/neradha/my-other-package.git
  Cloning https://github.com/neradha/my-other-package.git to c:\Users\my_username\appdata\local\temp\pip-install-07f6_gmu\my-other-package_9a293f5066d04cd58d17ae56a3ec609b
  Running command git clone -q https://github.com/neradha/my-other-package.git 'C:\Users\my_username\AppData\Local\Temp\pip-install-07f6_gmu\my-other-package_9a293f5066d04cd58d17ae56a3ec609b'
Collecting my-package@ git+https://github.com/neradha/my-package.git
  Cloning https://github.com/neradha/my-package.git to c:\users\my_username\appdata\local\temp\pip-install-07f6_gmu\my-package_d48d542db6334859919b60e7dadc5ea6
  Running command git clone -q https://github.com/neradha/my-package.git 'C:\Users\my_username\AppData\Local\Temp\pip-install-07f6_gmu\my-package_d48d542db6334859919b60e7dadc5ea6'
INFO: pip is looking at multiple versions of my-package to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install my-package and my-package 1.0.1 (from C:\path_on_my_pc\my-package) because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested my-package 1.0.1 (from C:\path_on_my_pc\my-package)
    my-other-package 1.0.1 depends on my-package 1.0.1 (from git+https://github.com/neradha/my-package.git)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

Pip should treat packages with the same name and same version as being identical

I agree in principle with this, but we've had complaints in the past precisely because of cases where pip does this. For example if the user updates the code and then does pip install ., they don't expect pip to say "nothing to do, . is the same version as is currently installed".

So getting the behaviour right in all the various cases is a lot more difficult than the "obvious" principle would suggest 🙁

[piotr-dobrogost]

@neradha
You might find disucssion at #8711 relevant.

[uranusjr]

Merging this into #5780 (comment). We decided to not refetch if the URL does not change, so this would work (and for those who want to update the code should use --upgrade).