Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: update expected wheels when using auditwheel >= 4 #633

Merged
merged 3 commits into from
May 9, 2021

Conversation

mayeut
Copy link
Member

@mayeut mayeut commented Apr 3, 2021

auditwheel >= 4 has major changes that needs to be reflected in cibuildwheel tests:

  • auditwheel always outputs the PEP 600 platform tag (e.g. foo-1.0-cp37-cp37m-manylinux1_x86_64.whl becomes foo-1.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl)
  • auditwheel now only outputs 1 wheel. Compatibility with a previous policy is now reflected in the wheel filename (e.g. instead of 2 wheels foo-1.0-cp37-cp37m-manylinux1_x86_64.whl & foo-1.0-cp37-cp37m-manylinux2014_x86_64.whl, we get 1 wheel foo-1.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl). Opting out of previous policy compatibility is done by using the --only-plat flag.

@henryiii
Copy link
Contributor

henryiii commented Apr 3, 2021

Ouch! This means that all older versions of pip will not be able to load wheels! Is there a way to get the old name back? The main reason to produce manylinux1 (and manylinux2010, mostly) is for old pip versions. The double-tagging seems useful, but not the name change to something pip 9, 10, ..., 20? cannot load.

@mayeut
Copy link
Member Author

mayeut commented Apr 3, 2021

This means that all older versions of pip will not be able to load wheels! Is there a way to get the old name back? The main reason to produce manylinux1 (and manylinux2010, mostly) is for old pip versions. The double-tagging seems useful, but not the name change to something pip 9, 10, ..., 20? cannot load.

@henryiii, they will be able to load those per the wheel spec (and I verified this).

@henryiii
Copy link
Contributor

henryiii commented Apr 3, 2021

How do they know how to load a wheel with a name on a spec that was invented after pip was released? (Not saying they don't, just have had to fight to get macOS wheels to have the right names (pip 21.0.1+ only), so wondering how this works)

@mayeut
Copy link
Member Author

mayeut commented Apr 3, 2021

Per the wheel spec, the platform tag shall be split by ..
Meaning the wheel will have:
Tag: manylinux_2_5_x86_64
Tag: manylinux1_x86_64
Tag: manylinux_2_17_x86_64
Tag: manylinux2014_x86_64
If any one is valid, it can be installed. Older pip will disregard manylinux_2_5_x86_64, manylinux_2_17_x86_64 & manylinux2014_x86_64 because it doesn't know them. It will consider manylinux1_x86_64 valid and thus install the wheel.

@henryiii
Copy link
Contributor

henryiii commented Apr 3, 2021

Ahh, I missed that the the 2010 and such tags were in the lists. That's great!

@Czaki
Copy link
Contributor

Czaki commented Apr 3, 2021

old macos wheels has multiple platform tags. Maybe because of this pip still could handle it.

@henryiii
Copy link
Contributor

henryiii commented Apr 3, 2021

This was my own confusion. I though it was making foo-1.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux_2_17_x86_64.whl , not foo-1.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl. Picking the dot-separated tags out by eye is a little hard.

@joerick
Copy link
Contributor

joerick commented Apr 23, 2021

Nice one, thanks for checking this @mayeut !

@mayeut
Copy link
Member Author

mayeut commented May 8, 2021

Force installing auditwheel in previous iterations of the PR masked the fact that we'd need an updated PyPy image as well (I'd really like to avoid a ugly hack for PyPy, even if just for testing).
Might relate to/depend on #629 at some point.

Need to wait for pypy/manylinux#16

@henryiii
Copy link
Contributor

henryiii commented May 8, 2021

Maybe the tests should support either version for a while to ease the transition. Maybe use packaging.utils.parse_wheel_filename() from packaging 20.9? Or check for one of two lists of names.

Then we can try to work on moving to version 4 uniformly if possible, as it would be nice to have the names match, but if PyPy was the odd one out for a bit, we might be able to live with that?

mayeut added 2 commits May 8, 2021 19:48
PyPy3.7 7.3.4 has been fixed and does not require the patch anymore.
@mayeut mayeut marked this pull request as ready for review May 8, 2021 17:50
@mayeut
Copy link
Member Author

mayeut commented May 8, 2021

Now that pypy/manylinux#16 has been merged, workarounds are limited to test/test_docker_images.py because auditwheel 4 is not installed yet on the dockcross images.
I had to update all images but updated all dependencies using the update script.
Given that update, also tweaked PyPy patching to only apply to PyPy3.6 while waiting for #629 to just drop PyPy3.6.

@henryiii
Copy link
Contributor

henryiii commented May 9, 2021

I need this in for #666

@henryiii
Copy link
Contributor

henryiii commented May 9, 2021

This only touches tests and standard dependency updates, so I think it's safe to merge.

@henryiii henryiii merged commit a972ee9 into pypa:master May 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants