Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
pipenv lock hangs. It really does. #3827
For a common setup consisting of scipy, pandas and numpy, here's the list of artifacts:
Artifacts Queued for Hash Retrieval
That's a lot of sequential network round-trips to wait through, and most of these are for different platforms than the installation. But that's not all.
Each artifact is passed to
The pypi artifact urls includes the sha256 hash, parsed into
The problem disappears if the user is patient enough to wait it out, or if the connection is fast. However, it seems most users are surprised by the delay (me included), and expect it to be more or less instantaneous once the packages are installed.
As a quick verification, I patched the
changed the title
pipenv lock downloads every possible artifact for every pyver/arch of all packages
Jul 7, 2019
Build times absolutely affect
Dependency resolution is an NP hard problem, there is no hack or easy trick around this, and in python it is also a problem that sometimes requires building artifacts. If you have identified and can avoid extra downloads that is excellent, but I do want to be clear: building is often a part of locking.
@jackiekazil apologies if you felt invalidated by the previous responses, we are all definitely aware that locking is slow, and as I mentioned in the other threads on the topic I agree that there are likely multiple downloads occurring but am not precisely sure where and would need to see debugging info to make any progress, so if the accompanying PR here addresses that it is awesome
my very first installation by pipenv got stuck by stopless "Locking", it just a small package and has already been install succeeded.
1 hour later, it was still there... annoying.
(posting this in the hopes that some reports of hanging are due to the same issue I encountered)
After multiple attempts to debug this, I kept seeing the same package causing problems (in my case, it was configparser, but I don't believe that's particularly relevant). I managed to replicate how pipenv runs resolver.py and recreated that environment in order to run the resolver.py under pdb. Under those conditions, I found that resolver.py would get stuck sitting in a sleep loop buried in a call stack that looked like it was trying to do some lockfile operations. When I looked at the directory it was trying to create in order to acquire the lock, it became clear what the problem was. The directory already existed and, in fact, had been on my filesystem for months (the length of time I have been having issues).
So in short, once I found:
I moved the that subtree to another location and then was able to successfully run pipenv lock again.
So for Mac OS, something like 'find ~/Library/Caches/pipenv/http | grep lock' might help highlight issues of this sort.
Well, I really cannot understand this.
Long-time ago, pypi doesn't contain metadata of the packages. There is no other choice but downloading all packages to calculate the hash. But it is 2019 now, pypi provides what you need.
In the past, I recommended this tool to others. Said "Oh, this is the next generation of pip.", "It is perfect, it not only manage virtual envs for you but also lock your dependencies.", "Just use it." ...
had the same problem. no error, verbose does nothing. really frustrating.
the ** needs globstar