Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upOn PyPy, binary wheels broken on pip==20.0.1 #7629
Comments
This comment has been minimized.
This comment has been minimized.
|
Same problem here, had to downgrade to pip 19.3.1 |
This comment has been minimized.
This comment has been minimized.
|
FWIW, I just tried and |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Same issue as in #7620 |
This comment has been minimized.
This comment has been minimized.
|
@chrisoro I think that's a different issue related to an ImportError. This bug is about pip not recognizing some supported binary wheel platforms. |
This comment has been minimized.
This comment has been minimized.
|
Thanks for the git bisect @neishm! I was pretty sure it's because we switched to |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@pradyunsg FTR this koalaman/shellcheck#1803 also looks like an instance of this issue but under regular CPython, not PyPy. |
This comment has been minimized.
This comment has been minimized.
|
I don't think so? If anything, it's related to #7626. |
This comment has been minimized.
This comment has been minimized.
|
Ah, right. Overlooked it. |
This comment has been minimized.
This comment has been minimized.
|
I think the python tag change from pp272 to pp27 is desirable, and in line with the discussion in pypa/packaging#233 and pypa/packaging#184. The pip19 python tag "pp272" is "pp" for pypy, "2" for python2, and "72" for pypy7.2.x. The python tag "pp27" is "pp" for pypy and "27" for python 2.7. Going forward, the new tag is more correct, especially as we start to support python3.6 and 3.7. Under the old naming scheme, both python3.6 and python3.7 would have the same python tag. How to deal with the transition period? There are very few PyPy wheels on PyPI, most of them are on https://antocuni.github.io/pypy-wheels. Perhaps that site could host pip19 and pip20 wheels for a while, doubling the number of wheels there. Once the pip20 release stabilizes, PyPy could bundle pip20 with PyPy. |
This comment has been minimized.
This comment has been minimized.
|
Whether there are wheels on PyPI or not, some users will be using binary wheels as part of their distribution system because it's significantly better than requiring that the end user compile the packages (or those end users may not have the ability to compile them). This is what I do with my CI, and have discovered today that the system that was working yesterday is no longer working, and (according to the message) the wheels that I'm trying to install are no longer supported. I can understand the need to differentiate between versions of Python for which the binary wheels were built for, but invalidating all existing versions seems to be... unhelpful. If it were me, and you were intent on providing a differentiated tag by the versions and needed to keep old wheels working, I'd be thinking about making '2' be an alias for '27' in the pip 20 series on reading, and assume that if wheels are created with the '27' they're only for 2.7. Of course, this also means that the creation of wheels should explicitly allow for the creation of the tags with '2' present if you want to keep python 2 working. I realise that it was stated that 'A future version of pip will drop support for Python 2.7', but did not quite expect that almost immediately after the 'end of life' date it would stop working. |
This comment has been minimized.
This comment has been minimized.
|
Also note that manylinux will not be checked for PyPy. |
This comment has been minimized.
This comment has been minimized.
|
@gerph it seems pinning pip and wheel (and for that matter all dependencies) versions would be prudent. Maybe when pypy binary wheels go more mainstream and the infrastructure to produce them becomes better tested, that restriction could be lifted, but it seems to me to be good practice even for CPython. |
There was an issue recently introduced with pip 20.0.1 that prevents binary wheels from being installed. This will result in failed setup of angr-dev. Until it gets resolved, simply use a slightly older version of pip. More info: pypa/pip#7629
There was an issue recently introduced with pip 20.0.1 that prevents some binary wheels from being installed. This will result in failed setup of angr-dev. Until it gets resolved, simply use a slightly older version of pip. More info: pypa/pip#7629
There was an issue recently introduced with pip 20.0.1 that prevents some binary wheels from being installed. This will result in failed setup of angr-dev. Until it gets resolved, simply use a slightly older version of pip. More info: pypa/pip#7629
This comment has been minimized.
This comment has been minimized.
|
CMake installation via pip is broken with pip version >= 20 because of this issue. I am not sure how prevalent it is, there are wheels like that for cmake that simply use pip as a platform and channel to distribute their manylinux binaries. These packages do not actually come with any Python or Python ABI requirements. Maybe it makes some sense to add new tags that makes the intention and requirements of these style of packages explicit. |
This comment has been minimized.
This comment has been minimized.
|
That's #7626. |
This comment has been minimized.
This comment has been minimized.
|
Explicitly noting the origin of the bug here: "pp272" is not a valid Python tag according to the wheel spec (https://www.python.org/dev/peps/pep-0425/#details), but the wheel project and pip presumably used to generate their PyPy Python tags the same way, so the issues cancelled out. While it isn't nice from a code readability point of view, the most user friendly fix would likely be to have a pass over the tag list in pip that added back the old (incorrect) "pp272" tags after the various "ppXY" tags. |
This comment has been minimized.
This comment has been minimized.
|
@ncoghlan's suggestion sounds good to me. |
This comment has been minimized.
This comment has been minimized.
|
@neishm I believe the problems you were seeing on CPython would have been due to #7626 rather than this issue. This one relates specifically to differences in the way that pip 20 and pip 19 calculate the set of supported tags on PyPy (example running on PyPy 7.0.0):
@pradyunsg The approach I'm taking is to add a |
pip used to calculate the interpreter tag for PyPy (and other non-CPython implementations) incorrectly. This was fixed as part of the migration to using packaging.tags to generate the list of potentially compatible wheel tags for a target platform. However, there are published PyPy wheels that use the old tag format, so this adds in a compatibility pass that accepts additional wheel tags in cases where pip versions prior to 20.0 previously did so.
See some recent issues filed with these releases, e.g. pypa/pip#7629
This comment has been minimized.
This comment has been minimized.
See some recent issues filed with these releases, e.g. pypa/pip#7629
This comment has been minimized.
This comment has been minimized.
|
OK, having attempted to implement this by changing the set of platform tags generated when running on PyPy, and having looked into more of the history of the custom PyPy wheel tags, I no longer think modifying Instead, now that wheel 0.34.0 has been released to build future PyPy wheels using the standard tagging scheme, I think it will be more fruitful to instead insert an extra tag into the compatibility set when
@antocuni I know sys.version still reported 3.5 in PyPy 7.0.0 (as that's the version in the Fedora 30 repos), but am I right that it switched to reporting 3.6 in PyPy 7.2.0, when the Python 3.6 support left beta? [1] There are enough full mirrors of PyPI running that the following should give all the relevant PyPy wheel interpreter tags:
[2] https://doc.pypy.org/en/latest/index-of-release-notes.html |
This comment has been minimized.
This comment has been minimized.
PyPy 7.0.0 was a "triple release" [1]: we released a PyPy2.7, a PyPy3.5 and a PyPy3.6: I suppose that fedora decided to include only 3.5 because 3.6 was marked as "alpha". After that release, we dropped support for 3.5 so PyPy 7.1, 7.2 and 7.3 only support 2.7 and 3.6 Note that we might decide to do the same in the future when 3.7 will be in an alpha state. But I think that now that PyPy reports the correct tag, it should no longer be an issue [1] https://morepypy.blogspot.com/2019/02/pypy-v700-triple-release-of-27-35-and.html |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Rather than relying solely on string manipulation as I suggested in my design sketch above, the updated PR ended up instead using a table of version thresholds for PyPy3. Extracting that data table from the current PR:
|
Environment
Description
If you try to install a manylinux wheel (such as the ones produced here), pip 20.0.1 complains:
pip==19.3.1installs it correctly.Expected behavior
I would expect the wheel to be installed correctly :)
How to Reproduce
First, download and install PyPy 7.2.0, and install pip==19.3.1:
Then, download and install a manylinux wheel: it works correctly:
Finally, upgrade to pip 20.0.1 and try again:
I tried it with PyPy 7.3.0 as well, same result.
More info
I tried to debug it a bit. I managed to reduce the difference to this snippet:
With pip==20.0.1, I get this:
Note that "supported tags" start with
pp27-, while "file tags" start withpp272-, soWheel.supportedreturnsFalse.