Start Python 3.12 migration with the release candidate

[xhochy]

Building upon #1499 and #1629 (cc @h-vetinari), I would like to start another approach and build Python packages with the release candidate before the final release.

How to do this

1. Get a 3.12rc build and upload it to conda-forge/labels/python_rc. By not having the RC on the main label, users will not be able to get Python 3.12 environments by chance, only if explicitly requested. We have a green build for rc2 ready at Python 3.12.0rc2 python-feedstock#636
2. Create a Python 3.12 migration that also adds this channel for Python 3.12 builds
3. Write a script that lists all yet built Python 3.12 builds. This is a safety net if something goes really wrong.
4. Start the migration and work on it
5. Wait until 3.12 is released
6. Build 3.12 final and push it to the main channel.
7. Remove the special channel from the migration

What if things go wrong?

We can easily mark builds as broken. In the case that this really happens, we have built a script as part of step 3. We would simply run this.

Will end-users end up with pre-release Python 3.12 environments?

No, the built packages will be all uploaded to the main channel except the one for Python itself. Thus python=3.12 won't be solvable until the final release has been built. The packages in CI can be built because there we are adding an additional channel.

What happens if there will be ABI breakage between the RC and the final release?

This is a very unlikely case and as outlined in numpy/numpy#19630 between the RCs and the final release are stricter requirements for changes than between patch releases.

If this still happens, we would simply mark all existing builds as broken using the script from point 3 and restart the migrator.

[h-vetinari]

One thing that's unrelated to the rc, but related to 3.12, is that - AFAICT - we haven't yet really discussed the implications of PEP602 for conda-forge¹. Key point being that Pythons versions which had been released in an 18 months cadence are fading away, while being replaced by new versions in a 12 months cadence.

We've now reached the new steady-state (one new release per year, one release EOL per yer), because 3.8 was the first to start the October-only releases, and it's been yearly ever since.

If we keep going as recently (drop oldest CPython when we add the newest one) this would mean that we'd be dropping support for 3.8 a full year before its upstream 5 year EOL, which is probably going to cause some grief.

The math is against us there: yearly releases + 5 years of upstream support means that if we want to match that, we have to - on average - also build for 5 CPython version everywhere. Or live with the fact that we drop support after 4 years... (the slow coming-of-age of the limited API may help a bit, but probably not by much).

Footnotes

1. zero mentions in this repo, for example ↩

[xhochy]

Small update here:

1. The rc of python has been built and merged to conda-forge/label/python_rc.
2. The migration is ready to merge: Add migration for Python 3.12 conda-forge-pinning-feedstock#4914 I would merge it tomorrow morning CEST if the following two PRs get merged beforehand (to unblock it)
3. Add 3.12 to conda-build-config python_abi-feedstock#22
4. Add Python3.12 migration cross-python-feedstock#74

[xhochy]

Started the migration: conda-forge/conda-forge-pinning-feedstock#4914

[jakirkham]

Since the migration is now in progress, can we close this issue? Or is there more still to do here?

[xhochy]

The last step Remove the special channel from the migration has been done, so closing.

[jakirkham]

Thanks Uwe! 🙏

[h-vetinari]

Some more xrefs for posterity:

• Add all channel_sources conda-forge-ci-setup-feedstock#279
• Variants with multiple channel_sources conda-forge-ci-setup-feedstock#280
• Add _python_rc staged-recipes#24067
• Add runtime _python_rc dependency for rc builds and publish to main python-feedstock#648