Release 24.1

[sbidoul]

It's April already.

Any @pypa/pip-committers up to take the RM hat this month?

[pfmoore]

Do we expect the packaging upgrade to be in 24.1? Because I don't have the bandwidth to deal with that change.

If the packaging upgrade isn't included, I could do it - but looking at the news fragments, it doesn't look like there's much in the release - do we even need a release? (I'm assuming we don't get a flurry of people merging changes to get them into the release - I don't have the bandwidth for that, either!)

[pradyunsg]

Do we expect the packaging upgrade to be in 24.1?

Yes.

---

If no one else has bandwidth, I'd be willing to be the RM for this one.

I expect we're going to have a beta period, to get some user feedback on the legacy version removal situation (thank you to @sbidoul for doing most of the work on #12300!) and to also make follow up fixes we'd likely need.

[pfmoore]

OK, if we're putting the packaging upgrade in, I'll leave it with you. I'm happy to help where I can, so feel free to ping me if you think I can help. I'll try to find some time to review @sbidoul's PR in the next day or two.

[sbidoul]

Regarding a beta release, I'm not entirely convinced it will help (my instinct says not enough people will notice/care), but that's up to the RM, of course ;)

[notatallshaw]

I would test a beta and I'm sure @potiuk would as well, I would also try to spread the word on social media as much as possible. However, I will say, the one time that pip recently had a beta it didn't get much exposure.

On release though, I will again mention that resolving is not sound #12317 and I have a fix here if anyone has time to provide any more feed back sarugaku/resolvelib#144

Also this PR seems fairly small and would be a great performance news item for Pip #12453 (and more complex but would be a big UX improvement news item, would be #12388 and #12404 if they are considered close to ready)

[NeilBotelho]

I'd say #12388 is mostly done, but I'm not sure if you would want to merge it into main by itself. It only adds support for parallel downloads without progress bars. #12404 adds support for parallel progress bars, but I'd like some more feedback on that before I'd be ready to call it done.

[notatallshaw]

A summary on macos runners which are currently blocking CI runs:

1. macos-latest has been updated to macos-14 running on ARM from macos-12 running on x86
2. The subversion package is not available by default and must be installed on macos-14
3. Until a couple of days ago Python 3.8 and 3.9 weren't available on macos-14
4. Old versions of virtualenv do not work on macos-14, and pip test suite currently pins Python 3.8 and 3.9 to virtualenv<20
5. Removing this restriction reveals 1 failure "Tests_UninstallUserSite::test_uninstall_editable_from_usersite" on macos-14 Python 3.8 and 3.9

So it needs to be decided whether to keep running x86 as well as ARM (#12593) or just stick with "latest" which is now ARM (#12617). And someone either needs to fix this test (I don't know what the issue is and have no way to reproduce locally) or whether to exclude Python 3.8 and 3.9 from MacOS ARM.

[notatallshaw]

I'd also like to nominate #12657 for 24.1, it's very simple, and I've measured a significant performance improvement on large number of packages to resolve, and even a slight perfornance improvement on small number of packages to resolve.

[pradyunsg]

Oh wow, this month has flown by. I'm catching up on my RM duties now. 😅

[vx1920]

I did report small issue while ago: useless warnings in "pip list --outdated --verbose".
Possible fix with adding two lines to commands/list.py was described, it will allow suppress warnings optionally.
See #12540
Should I create pull request ?

[colesbury]

I know this was asked by Paul above and answered, but is the packaging upgrade still slotted to be in the 24.1 release? (From the outside, it's hard to tell how close or far away the PR is to being ready.)

Do you expect the 24.1 release before the Python 3.13 beta 1?

[sbidoul]

I posted a status update on the PR: #12300 (comment)

[notatallshaw]

As it seems there's very likely to be a beta that hopefully will get some wide coverage, I would like to make the case for pip maintainers to be a little bit more agressive merging approved open PRs that are in a good state:

https://github.com/pypa/pip/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved+draft%3Afalse+status%3Asuccess+sort%3Aupdated-desc+

I appreciate pip is under resourced and generally conservative in accepting PRs, but having a long open approved unmerged PR can be a little demoralizing to potential future contributors.

[pfmoore]

I've moved a few of those onto the 24.1 milestone, where I think they are clear-cut ones that should be merged. The others I think might need a little more consideration than I have time for right now. (I'm not arguing against them, though).

[pradyunsg]

OK, we're almost all set for 24.1b1 to be released today. I have one outstanding PR to merge and then will cut a release after grabbing lunch today.

[pradyunsg]

https://pypi.org/project/pip/24.1b1/ is up. I'm not updating get-pip.py or filing a PR to CPython with an updated wheel for this.

[pfmoore]

Are you going to post an announcement on discuss.python.org? It might be a good way to attract a few more beta testers.

[pradyunsg]

Yep, I plan to do that along with a blog post that serves as a call for beta testing.

That'll all have to be after my work day today though, since pip maintainance isn't a part of my day job anymore.

[potiuk]

FYI. I can confirm that the most complex case with installation and eager upgrade for latest airflow version with 24.1b1 works fine. Not super comprehensive, but passes as "smoke test" IMHO

[pradyunsg]

@potiuk Is it meaningfully quicker? :)

[potiuk]

Not too scientific but looks like ~ 15% (~550 seconds -> ~470 seconds) - on a very small sample.

[notatallshaw]

Not too scientific but looks like ~ 15% (~550 seconds -> ~470 seconds) - on a very small sample.

Roughly what is that running?

Anything that gets stuck doing a bit of backtracking should see more significant performance gains. But if the versions here are all tightly constrained that is about in line with what I would expect.

[potiuk]

Roughly what is that running?

Roughly this:

curl -fsSL https://github.com/apache/airflow/archive/main.tar.gz | tar xz -C /tmp/tmp.18llqGZdHv --strip 1
pip install --root-user-action ignore --editable '/tmp/tmp.18llqGZdHv[devel-ci]'

[pradyunsg]

OK, we have 2 more PRs to land and it looks like GHA broke 3.13.0b2's tarball used by setup-python.

[henryiii]

Probably actions/setup-python#886 ? You could manually set 3.13.0b1 for a bit to unblock. Looks like 3.12.4 might have the same issue.

[hugovk]

Pin using "3.13.0-beta.1", but it will need some mangling to work with the nox -s test-${{ matrix.python }} call.

[henryiii]

I’d mangle the setup-python call instead. Easier and doesn’t change the job name. Something like ${{ matrix.python == '3.13' && '3.13.0-beta.1' || matrix.python }}.

[mayeut]

A workaround shouldn't be needed anymore. Broken versions were removed from the manifest.

[pradyunsg]

And, we're green! Thanks all!

[pradyunsg]

https://pypi.org/project/pip/24.1b2/ is up!

[edgarrmondragon]

24.1b2 works great for us and fixes the problem described in #12613 (comment) 🙌

[potiuk]

Also Airlfow build works nicely with pip set as default installer and 24.1.b2 apache/airflow#40184 👍

[pradyunsg]

Alright, #12777 is up. I'll push all the relevant knobs tonight / tomorrow night depending on my availability.

[pradyunsg]

Alrighty, all done with the button clicking for now.

• https://pypi.org/project/pip/24.1/ is up.
• Update to 24.1 get-pip#213 landed.
• https://discuss.python.org/t/announcement-pip-24-1-release/56281 is up.
• I haven't filed a CPython PR, but will file one next week.

[hugovk]

• I haven't filed a CPython PR, but will file one next week.

If possible, it'd be great to have it in time for Tuesday's 3.13.0b3.

https://peps.python.org/pep-0719/#schedule

[pradyunsg]

@hugovk Thanks for that nudge -- made the PR now, given that there's not really any major showstoppers AFAICT with this release.

[notatallshaw]

I believe #12781 needs more attention.

My cursory analysis is that the new tar extraction logic fails for tars which contains hard links, but it wasn't clear to me why, and neither the regular Python extraction or the data filter extraction failed, perhaps I am wrong in my analysis.

[pfmoore]

I believe #12781 needs more attention.

See my comments there. I'm not sure we want to rush a fix for 24.1.1, as I think the subtleties involved warrant deeper consideration, and a "quick fix" might just leave us with further issues.

[pradyunsg]

FWIW, I guess I should mention this explicitlt: main is open for non-bugfix changes. I'll cherry pick for any follow up bugfix releases we need.

[pradyunsg]

Cutting a 24.1.2 with #12787 and #12799 cherry-picked. #12831 is the relevant PR.

[pradyunsg]

OK, assuming I don't see any fires in the next few minutes about 24.1.2 (uploaded to PyPI and get-pip.py has also been updated), I'm gonna say this doozy of a release is done with.

And, I'll move on toward making 24.2 next weekend. 😅

[pradyunsg]

Ok, gonna say about 25 minutes is enough time.

Thanks everyone for helping make this release happen -- this was a surprisingly long release cycle to manage. We had hiccups and a bunch of technical details needed to be figured out and yet this went by without major disruption... and, overall, quite smoothly!