pip wheel -e git+https:// does a git export instead of git clone

[sbidoul]

Environment

• pip version: 20.3
• Python version: 3.8 (probably irrelevant)
• OS: linux (probably irrelevant)

Description

pip wheel --no-deps -e git+https://github.com/pypa/pip-test-package#egg=pip-test-package used to keep a git clone in the src directory. Now it keeps something there, that does not have a .git directory in it.

Analysis

In progress, but a quick git bissect shows that the issue originates in the same area as #9122.

[sbidoul]

This matters because if the package being built relies on setuptools_scm's git file finder, it does not work anymore.

[uranusjr]

Does this only affect wheel, or also install?

[sbidoul]

install works fine. The problem comes from here:

pip/src/pip/_internal/operations/prepare.py

Line 573 in 94b89c9

req.update_editable(self.download_dir is None)

which was changed in #8843 (the merge of wheel_download_dir and download_dir). The changed argument is obtain which drives whether pip does a vcs clone or vcs export in update_editable.

Now it does a vcs export in wheel mode too, whereas before it did that in download mode only.

[sbidoul]

So this bug has indeed the same origin as #9122, which is that _download_should_save has been replaced by a test on the presence of download_dir, which is not equivalent.

[sbidoul]

I suspect this clone vs export issue can also have implications on dependency resolution, for wheel and download alike. Indeed, even in download mode, if the vcs information is not present, setuptools_scm may yield incorrect version information, influencing the resolver outcome.

[sbidoul]

So, a radical approach could be to always vcs clone and never vcs export, which would fix this bug, but change slightly how pip download of editable requirements works, as follow.

	20.2.4 src has .git dir	20.2.4 zip has .git dir	20.3.1 src has .git dir	20.3.1 zip has .git dir	proposal src has .git dir	proposal zip has .git dir
pip download git+https://	n/a	yes	n/a	yes	n/a	yes
pip download -e git+https://	no	no	no	no	yes (this is an improvement)	yes (is this change a problem?)
pip wheel git+https://	n/a	n/a	n/a	n/a	n/a	n/a
pip wheel -e git+https://	yes	n/a	no (this bug)	n/a	yes (this bug is fixed)	n/a

What do people think?

[sbidoul]

I'm assigning this to the 20.3.4 milestone it this is a regression. #9274 implements the table above. See #9338 for the root cause analysis and discussion about possible approaches.