Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

pip install fails to locate package, but easy_install can find it #913

Closed
stsci-sienkiew opened this Issue Apr 26, 2013 · 12 comments

Comments

Projects
None yet
3 participants

I have created a package on testpypi that pip cannot install but easy_install can.

The problem appears to be that pip does not understand the page at this URL:

https://testpypi.python.org/pypi/pipbugdemo01

It shows two versions of the package that are available. easy_install can install the package, but pip says "Could not find any downloads that satisfy the requirement pipbugdemo01".

If I tell pip a version number with == then it can find it, presumably by skipping directly to the page. If I tell pip a minimum version number with >= it cannot.

If I use the pypi interface to "hide" one of the versions (leaving only one version unhidden), pip is able to install the package. Since is is an option on pypi to show the availability of more than one version of the package, I expect pip would understand that format.

A detailed session log showing a reproduce-by sequence is included in the README for the package. See https://testpypi.python.org/pypi/pipbugdemo01/1.1

Showing only one version is not an acceptable workaround, since I am required to use >= in requires lists for another package. I can tell my users "you must use easy_install instead of pip", but that has other problems.

Contributor

embray commented Apr 29, 2013

Is it possible this is just an issue specifically with testpypi?

For example pyfits exposes multiple versions: https://pypi.python.org/pypi/pyfits

pip install python works fine for me.

Django shows tons of simultaneously available versions:

https://pypi.python.org/pypi/Django

pip install Django works fine too. So that's odd.

Contributor

embray commented Apr 29, 2013

Oh, I didn't realize the test package was using a non-PyPI download_url. I think download-url is typically expected to be a link directly to the source archive. I'm not sure though--they documentation on this is unclear. I think the fact that easy_install will still accept an HTML page as download_url and scan it for archives is nice behavior, but I don't think it's necessarily expected behavior either.

I confirm the iguananaut's observations about pyfits. I had no trouble with

pip install 'pyfits>=3.0'

I can still reproduce the problem with pipbugdemo01 from testpypi. I was assuming that pypi and testpypi are the same software. A superficial examination did not reveal any obvious difference in the structure of the two download pages.

I will try to reproduce on the real pypi.

I was NOT able to reproduce the problem on pypi.python.org. See the package pipbugdemo01, which I will leave around for a while to demonstrate and then delete when this issue is resolved.

When I view source on the web pages, I see at least line break differences between these two urls:

https://pypi.python.org/pypi/pipbugdemo01
https://testpypi.python.org/pypi/pipbugdemo01

Here is something more substantial:

testpypi: ::

96         <li><a href="http://sourceforge.net/tracker/?group_id=66150&amp;atid=513503">PyPI Bug Reports</a></li>

pypi: ::

113         <li><a href="https://bitbucket.org/pypa/pypi/issues">PyPI Bug Reports</a></li>

So, we can conclude that pypi and testpypi are not, in fact, running the same software.

I also found that I can install pipbugdemo01 from pypi, even though it appears to have about the same configuration as on testpypi.

https://bitbucket.org/pypa/pypi/issue/10/testpypi-and-strange-interaction-with-pip is a report of this bug on testpypi. I'm not sure if the problem is here or there, so I will leave both reports open.

I do, however, believe I am using download_url correctly as described here: http://pythonhosted.org/distribute/setuptools.html#making-your-package-available-for-easyinstall

If download_url were expected to be only the source distribution, there would be no way for easy_install to find the .egg files with architecture specific object files. I did not make egg files for this reproduce-by, but I do make Windows egg files for some of my packages, so I have to be able to list both the source and the binaries for download.

Contributor

embray commented Apr 30, 2013

The issue is not so much with parsing the version listing page. Rather, pip just doesn't know how to drill into that page. If you give it a try with -i https://pypi.python.org/pypi you'll find the same problem--by default pip is using https://pypi.python.org/simple which is easier to parse.

Additionally, although it should work to use -i https://testpypi.python.org/simple apparently testpypi isn't creating the necessary links there. It should look like this:

https://pypi.python.org/simple/pipbugdemo01/

but on testpypi there are no links:

https://testpypi.python.org/simple/pipbugdemo01/

Hrm :/

Owner

dstufft commented Apr 30, 2013

I'm pretty sure testpypi is running the code that implements phase 1 of PEP438. Basically there are 3 hosting modes. One that implements the status quo, one that continues to parse the long_description for links but the home page and download_url links are given a different rel attribute so as not to be fetched by pip/easy_install, and the final one does not automatically add any urls to the simple index besides the ones linking to files hosted on PyPI.

Part of the change also entails changing new projects to default to the most restrictive mode (for many reasons detailed in the PEP I believe), but allowing them to change which mode they use. It also provides a means for package authors to explicitly select external urls that will exist on the /simple/ index (and thus be eligible for install) but they must be direct links to installable items.

Owner

dstufft commented Apr 30, 2013

Going to close this ticket since this is intended behavior.

@dstufft dstufft closed this Apr 30, 2013

I'm confused. According to the cited PEP, "At completion of the first transition phase, all present-day existing release and installation processes and tools are expected to continue working." The bug report states that this condition is not being met. So, where would you say the problem is?

I can conjecture two possibilities:

  • in pypi itself
  • in my setup.py

If I should be doing something different, can you tell me where I can find out what? Should I write a program that performs the POST operation described in the section "API For Submitting External Distribution URLs"? If so, do you know if the production pypi server understands them yet?

Owner

dstufft commented Apr 30, 2013

That may be worded poorly. It means for existing projects. Since this is a "new" project (as far as testpypi is concerned at least) it has no existing release or installation process. The use of the modes allows a project to decide which method of discovery they want for external urls.

Ideally you should host the files directly on PyPI using ``setup.py sdist upload`. If you cannot or will not do that for one reason or another then yes you should be using the API for submitting External Distribution URLs. The Production PyPI server does not to my knowledge understand it yet. You can also use the webform to add or delete external distribution urls as well. Again this is not live on PyPI to my knowledge.

Contributor

embray commented Apr 30, 2013

"The first phase also will default newly-registered projects on PyPI to only serve links to release files which were uploaded to PyPI." I didn't know about this either but that seems to answer the question.

For pyfits I upload both to PyPI and to our local package index. In the past I've given out instructions on how to use the local index if PyPI is for some reason unreachable. I think that seems like an acceptable solution. There advantages to hosting the files on PyPI, not the least of which is that it makes it easier for them to be picked up by mirrors.

Owner

dstufft commented Apr 30, 2013

It actually makes it possible for them to be mirrored. Uploading to PyPI you grant people a license to redistribute (but not necessarily modify or use) your files which is what allows the mirrors to legally be allowed to grab those files and rehost them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment