New release with ability to load 10.9 Universal2 wheels on ARM

[henryiii]

The inclusion of #380 is critical for macOS ARM users - the "correct" name for most universal2 wheels is not supported by pip currently, but fixed in packaging master. Can we get an updated packaging and pip release? (I'll also push for a poetry release after packaging updates) Cibuildwheel is hoping to release support for Universal2 soon (pypa/cibuildwheel#484 (comment)), but the only options are to either a) ignore universal wheels and only produce x86_64 and arm64 (@ronaldoussoren would not like this, and 90% of the time I would agree), b) have a 10.9 x86_64 wheel and a 11.0 universal2 arm64 wheel, or c) dual-tag the universal2 wheel as 10.9 and 11.0 via rename hack.

[brettcannon]

@pradyunsg is there a pip release that's about to come out which would gate a release from here?

[pradyunsg]

I don't see why pip would gate this? It's usually the other way round. 🙃

We "just" update the dependency held inside pip before a release, so whatever the newest packaging version is, we'd use that. :)

[pradyunsg]

FWIW, pip 21.0 is due later this month. So, if we do get a release of packaging out this week, I can have pip vendor it in time for that. :)

packaging also goes to 21.0, for which I'd really really like us to include #376 in.

[brettcannon]

@pradyunsg I'm mainly asking so we don't release now, find out pip needs something for a release soon, and then we do another release (basically minimizing overhead 😄 ).

And I'm totally in support of pulling #376 into this.

[brettcannon]

@di @pradyunsg so what do you we want to do? Remove Python 2 support and release? Release and then remove Python 2?

[di]

I just approved #376 but also saw this comment:

The days for PyPy 2 might be numbered after packaging drops support #376 . Given they want to merge this before releasing a version that supports universal2 wheels, pypy2 might not be able to ever work properly on Apple Silicon.

This makes me think we should do one last release that supports both Python 2 and universal2 wheels.

[henryiii]

@mattip , this might be something you have an opinion on?

[pradyunsg]

Let's do one last release with Python 2 support, and then have one more immediately after that drops Python 2 support but nothing else. :)

[henryiii]

If you haven't tried it, maybe try running https://github.com/asottile/pyupgrade right after #376, too.

[mattip]

One way to look at pypy2.7 is as a minimal bootstrap intermediate in order to translate (compile) pypy3 from the source code, which is written in RPython (a restricted derivative of the Python2 syntax). Did that make sense? Through that perspective, it is not really important to the PyPy project to be able to use pip in pypy2.7 to install binary packages. We still need to be able to test our bootstrapped (pypy2.7) implementation, so we need pytest and hypothesis and a few other pure-python packages. These should all be package-py2.py3-none-any.whl and installable even on new hardware.

Bottom-line: @pradyunsg's plan sounds reasonable.

[henryiii]

@pradyunsg @pfmoore @ronaldoussoren Why was packaging not bumped before Pip 21.0? At cibuildwheel, we've been waiting for a release that can load properly named universal2 wheels before releasing building support for them since mid December (since we have to apply a hack to get them to even load to test them on AS). I explicitly opened this issue to try to ensure this patch made it into pip 21.

[henryiii]

It would also help Poetry, as they have been stuck without an update to packaging for a while, and if they can go directly to 20.1/20.9/20.8.1 that might be quite helpful for Apple Silicon users.

[pfmoore]

pip 21.0 was released with the latest version of packaging that's available (20.8). I don't know what the release schedule is for packaging, but when a new release is made available, it will be included in the next pip release after that. But we don't co-ordinate release timings between the projects, if that's what you're asking...

I wasn't aware of this issue when I cut the release, so sorry for that. As an admin point, if there's something that should be going into a pip release, it should be opened as an issue on the pip tracker and added to the relevant release milestone, then it won't get missed. Although in this case, this weekend was the last one I had free to do the release in January, so as packaging didn't ship anything in time, I'd have done the release anyway rather than defer it, because we were trying hard to get back onto our normal release schedule after delays in the later releases of 2020 for the resolver work.

[henryiii]

Can we please get a 20.8.1/20.9 release with #380 in it? It is needed to continue the wheelbuilding effort for Apple Silicon. (The only other PR before that was #377 for isort which is likely safe). Recently there was #387, which could wait till the next release if there's any chance it could cause problems (haven't looked at it in detail). Then #376 (drop old Python support) can go in and you can release 21.0 whenever you like. :)

[henryiii]

@pradyunsg @di Gentle ping. Any chance we can get the release before the weekend, so pip can possibly include it?

I don't like pinging too often, but I didn't ping on this release before and we missed Pip 21.0 and 20.3.4, and it looks like there may never be a 20.3.5.

[brettcannon]

We are trying to resolve #389 first, and then we will do a release (as for pip, that's a separate thing).

[henryiii]

Can't we get a patch release based before #387? That's adding a new feature, which is why there's an API discussion for it in #389; I just want an existing feature fix released so the Apple Silicon wheel building can continue.

[brettcannon]

@henryiii because we don't maintain a release or dev branch, so we would have to undo any commits to main we don't want to go out to give you your point release.

I obviously can't promise anything, but my guess is you will get a release in less than a week as I don't see #389 taking long to resolve.

[brettcannon]

I think all the blockers are now taken care of, so now it's just a matter of someone having the time to make the release. 😄

I can't promise anything, but I will see if I can do a release on Friday.

[pradyunsg]

Whoever is making the release please call it 20.9, because we've not had a 3 part version in a whole and there's something nice about bumping up the major version for the right reasons. 🙈

Also, please drop a comment here before starting the release process because race conditions from 2 people doing the same thing here would be bad. 🙃

[brettcannon]

It's rather ironic that when I tried to do the release last night to fix an issue related to macOS that my macOS laptop stopped working with WiFi 🙄

If I manage to get to the release today I will comment here before I start.

[brettcannon]

Looks like my lunch time got pushed out by an hour, so I'm going to give it a shot ...

[brettcannon]

https://pypi.org/project/packaging/20.9/

[henryiii]

This is fantastic, thank you very much!