Upgrading packages from 'git+ssh' not working

[aknapp]

Tried upgrading to pip version 1.1 today, and I'm having some difficulties when running pip with the upgrade command. Let me walk through this.

First off, my requirements file looks something like this (I should note that blah is just an internal project):

git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009#egg=blah

I'm able to get into my virtualenv and install the package just fine, with a command like this:

pip install -v -i http://some.internal.pypi.server/ -U --use-mirrors --mirrors "http://d.pypi.python.org" --mirrors "http://b.pypi.python.org" -r ~/test-deps.txt

That works. Now say, I want to upgrade some packages (say my requirements file has more packages in it). When I run the same command again (this time with verbose to get more output), I get this:

Using PyPI mirrors: http://b.pypi.python.org/simple/, http://d.pypi.python.org/simple/
Could not fetch URL http://some.internal.pypi.server/blah: HTTP Error 404: Not Found
Will skip URL http://some.internal.pypi.server/blah when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))Could not fetch URL http://some.internal.pypi.server/blah/: HTTP Error 404: Not Found
Will skip URL http://some.internal.pypi.server/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))Could not fetch URL http://d.pypi.python.org/simple/blah/: HTTP Error 404: Not Found
Will skip URL http://d.pypi.python.org/simple/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))Could not fetch URL http://b.pypi.python.org/simple/blah/: HTTP Error 404: Not Found
Will skip URL http://b.pypi.python.org/simple/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))Could not find any downloads that satisfy the requirement blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))No distributions at all found for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))Exception information:Traceback (most recent call last): File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/basecommand.py", line 104, in main
status = self.run(options, args)
File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/commands/install.py", line 245, in run requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/req.py", line 920, in prepare_files
req_to_install, self.upgrade)
File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/index.py", line 157, in find_requirement
raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))

(Sorry if that turns out formatted horribly). Now, if I substitute out -U with -I in the above command, it works (because it just reinstalls the packages). Also, downgrading to 1.0.2 fixes the problem, and it doesn't error out when trying to upgrade. Any ideas?

[aknapp]

Fixed up the stacktrace, sorry about that.

Using PyPI mirrors: http://b.pypi.python.org/simple/, http://d.pypi.python.org/simple/
Could not fetch URL http://some.internal.pypi.server/blah: HTTP Error 404: Not Found
Will skip URL http://some.internal.pypi.server/blah when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
Could not fetch URL http://some.internal.pypi.server/blah/: HTTP Error 404: Not Found
Will skip URL http://some.internal.pypi.server/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
Could not fetch URL http://d.pypi.python.org/simple/blah/: HTTP Error 404: Not Found
Will skip URL http://d.pypi.python.org/simple/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
Could not fetch URL http://b.pypi.python.org/simple/blah/: HTTP Error 404: Not Found
Will skip URL http://b.pypi.python.org/simple/blah/ when looking for download links for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
Could not find any downloads that satisfy the requirement blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
No distributions at all found for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))
Exception information:
Traceback (most recent call last):
  File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/basecommand.py", line 104, in main
    status = self.run(options, args)
  File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/commands/install.py", line 245, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/req.py", line 920, in prepare_files
    req_to_install, self.upgrade)
  File "/home/andrew/.env/temp/lib/python2.6/site-packages/pip/index.py", line 157, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for blah from git+ssh://gitosis@some.server.interal.domain/compname/blah.git@176bb2cf5041152e7324de482e459b76c7b30009 in ./temp/lib/python2.6/site-packages (from -r /home/andrew/test-deps.txt (line 2))

[carljm]

Oops, looks like I broke that a while back. Fixed now, thanks for the report.

[aknapp]

So, I just tried this out, and it looks like it's still a little wonky. It doesn't fail out now, but for packages that we have a local git repo for (say, django, django-hosts, protobuf, etc..), it installs correctly the first time from our repos (on a bare virtualenv), but when I upgrade, it goes and fetches a version from pypi instead of from our git repo.

I just seems like the order is off. Any ideas?

Example:

We have a local copy of protobuf in a git repo. When I first install it from a requirements file, it works fine:

Downloading/unpacking protobuf from git+ssh://gitosis@internal.git.server/thirdparty/protobuf-python.git@a8c173e08fe262cbdb64989d11bbde32535598ea (from -r /home/andrew/dev/git/requirements.txt (line 2))
  Cloning ssh://gitosis@internal.git.server/thirdparty/protobuf-python.git (to a8c173e08fe262cbdb64989d11bbde32535598ea) to /home/andrew/.env/env/build/protobuf
  Could not find a tag or branch 'a8c173e08fe262cbdb64989d11bbde32535598ea', assuming commit.
  Running setup.py egg_info for package protobuf

Now, when I upgrade the environment, it does this:

Downloading/unpacking protobuf from http://d.pypi.python.org/packages/source/p/protobuf/protobuf-2.4.1.tar.gz#md5=72f5141d20ab1bcae6b1e00acfb1068a (from -r /home/andrew/dev/git/requirements.txt (line 2))
  Using download cache from /home/andrew/.pip-cache/http%3A%2F%2Fd.pypi.python.org%2Fpackages%2Fsource%2Fp%2Fprotobuf%2Fprotobuf-2.4.1.tar.gz
  Running setup.py egg_info for package protobuf

It goes out to pypi and downloads a newer version and installs it (our git repo version is 2.4.0a, and on upgrade it installs 2.4.1). Hope this helps.