Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
pip.index.py get_mirrors() is broken. 212,102 mirrors returned #605
To create the mirrors, it uses this....
It expects that the returned "last.pypi.python.org" is actually "g.pypi.python.org", which it uses to calculate the other mirror hostnames. This is the seed for the mirrors and because this is wrong, the whole mirroring is broken on a fresh installed VM when pypi.python.org is unreachable.
The list of resulting 4-character mirrors results in 212,102 hostnames which are all invalid mirrors. As you can imagine, pip takes a long time to try all of these.
If I query the CNAME with dig, then it actually comes back ok, so it seems calling gethostbyname_ex is the wrong thing?
;; ANSWER SECTION:
Are the mirrors cached somewhere? Once pypi.python.org comes up again we never see this issue again and the mirrors are correct.
I just forked and downloaded onto my Mac, and running "python setup.py test" hangs at "Test installing a package from the PyPI mirrors. ...".
I used dtruss and I can see this is hitting the same issue, so this is not limited to fresh VMs or Ubuntu (the VM I originally found it on).
This is what I see from Vancouver, Canada...
This is what I see from Japan...
When I'm on a machine in Vancouver or on VPN, all is good, but from here, Japan, it's returns the wrong thing and sends my CPU into a spin at hits 212k invalid mirror hosts.
I raised the fragility of this a while back on catalog-sig http://mail.python.org/pipermail/catalog-sig/2010-July/003150.html
Can you give more details on the OS and python versions for the boxes above
On 2012-07-18, at 7:45 PM, Paul Nasrat wrote:
Interesting read, thanks.
These are the versions, but it seems network related rather than OS. If I connect to the VPN, works on both. I have no other network issues that I'm aware of.
MacAir Lion 10.7.4