Pip saves vcs-dependency in zip-file with version

[l0kix2]

Noticed, that pip in 6 and 7 (maybe earlier) began to save cached vcs dependencies in zip files with version name.

So let's assume i have package mylib, in setup.py it has version 1.0 and i use it in my project like this

#requirements.txt
mylib==1.0

Once i made some branch with some features in mylib, so now i wanna test it within my project from branch, so i put

git+git://github.com/l0kix2/mylib.git@e3e3e6fd5716577e8a31c2223d16a6e175a8bfad#mylib

in my projects requirements.txt in some staging branch of my project.

After that i build my virtualenv like that (with files cache):

pip install --download=/tmp/.pip --find-links=file:///tmp/.pip --no-index --no-clean -r requirements.txt

So pip says

Saved /tmp/.pip/mylib-1.0.zip

And when i do same command in master of my project (where i have mylib==1.0) on same machine i got this zip file from cache, but it's not 1.0 at all, it is some fork of 1.0 version of lib with some changes made in branch.

I guess, pip should save file in cache with name like /tmp/.pip/mylib-e3e3e6fd5716577e8a31c2223d16a6e175a8bfad.zip to avoid such problems.

Any suggestions, workarounds are welcome?

[dstufft]

Can you verify that this is still happening in 7.0.1?

[l0kix2]

Hi, Donald. It seems, that behavour in 7.0.1 is the same. These are some steps for reproduce. Correct me if i'm doing something wrong.

% mktmpenv
% pip install pip==7.0.1
% (tmp-ddc00b79909dfd4)% pip -V
  pip 7.0.1 from /Users/l0ki/.virtualenvs/tmp-ddc00b79909dfd4/lib/python2.7/site-packages (python 2.7)

installing (downloading actually) from branch

(tmp-ddc00b79909dfd4) % pip install --download=/tmp/.testpip --find-links=file:///tmp/.testpip --no-index -e git+git://github.com/l0kix2/python-debparse.git@51b4c12d9917a23af43f752f8c8bc66d64a9cdd6#egg=debparse

Obtaining debparse from git+git://github.com/l0kix2/python-debparse.git@51b4c12d9917a23af43f752f8c8bc66d64a9cdd6#egg=debparse
  Cloning git://github.com/l0kix2/python-debparse.git (to 51b4c12d9917a23af43f752f8c8bc66d64a9cdd6) to /var/folders/3k/myxmp7wd2_jgwzft0g1gv9tc0000gn/T/pip-kjIuBa-export
  Could not find a tag or branch '51b4c12d9917a23af43f752f8c8bc66d64a9cdd6', assuming commit.
...
Saved /tmp/.testpip/debparse-0.1.0.zip
Successfully downloaded debparse

Let's check what was downloaded in debparse-0.1.0.zip

% unzip  /tmp/.testpip/debparse-0.1.0.zip

% cat debparse/__init__.py
# coding: utf-8

from .import deb_control

# testpip branch

(i added comment in __init__.py in testpip branch so i can say which code was installed)

Now we try to install package using version, telling pip we want it from cache

(tmp-ddc00b79909dfd4)% pip install --find-links=file:///tmp/.testpip --no-index debparse==0.1.0
Ignoring indexes: https://pypi.python.org/simple
Collecting debparse==0.1.0
Installing collected packages: debparse
  Running setup.py install for debparse
Successfully installed debparse-0.1.0

And let us check what was installed

(tmp-ddc00b79909dfd4)% cat $VIRTUAL_ENV/lib/python2.7/site-packages/debparse/__init__.py
# coding: utf-8

from .import deb_control

# testpip branch

So package was installed from /tmp/.testpip/debparse-0.1.0.zip which was downloaded at first time, but i expect, that package with specified pypi version shouldn't be matched with package cached with vcs installation.

It may seem quite complex workflow for installing, but i didn't come to it by myself, we use wrapper for this called https://github.com/paylogic/pip-accel, so it does this caching work while building our project virtualenvs, but it looks that it uses pip correctly in this case.

[dstufft]

Oh, wait I see. It's using pip install --download to build a folder full of files, and then it tries to install from those again using pip install --no-index --find-links. The behavior of pip here is correct then, we don't determine the version number of the project/file, that comes from inside the package. If they want to support arbitrary tags being independently identifiable they should have their setup.py adjust itself based on that.

[l0kix2]

we don't determine the version number of the project/file, that comes from inside the package.

So when installing pip just checkouts repository and then it treats it like local package and installs it like other packages pointed by filesystem path, am i understanding correct? And when using download it checkouts and makes sdist and puts it in cache, so vcs repo kinda not considered?
Maybe there is some workaround about specifying version in vsc url or something?