CIP-0008: toward a classic main-branch release pipeline

[lawrennd]

Why this thread

We want to move GPy off the current GitHub Release–gated wheel/PyPI path toward a classic main-centric release pipeline (one primary branch, tag → build all wheels including Linux aarch64 → publish), with a dry-run that does not require publishing a Release first.

Design is captured as CIP-0008 (Proposed) so it can still change before we Accept it for the next release:

• CIP (PR): Add CIP-0008: classic main-branch release pipeline (Proposed) #1227 — cip/cip0008.md
• Related packaging: Tracking: packaging residuals (matplotlib import + arm64 Cython wheels) #1209 (optional plotting + aarch64 wheels)

Please read the CIP Open questions and reply here — especially if you still cut or rely on releases.

Open questions (from the CIP)

1. Primary branch: stay on devel, or move default to main?
2. Retire the deploy branch from CI triggers?
3. Who can publish (branch protection / environment approval)?
4. Optional TestPyPI / pre-release gate before first aarch64 production upload?
5. Enforce tag ↔ Poetry version match in CI?

Asking for eyes

Tagging people who have historically maintained packaging, releases, or core GPy (please ignore if this is no longer your beat — a short “still relevant / not me” is enough):

• @lawrennd
• @mzwiessele
• @jameshensman
• @alansaul
• @mikecroucher
• @MartinBubel
• @nfusi
• @bobturneruk
• @zhenwendai

Aim: leave CIP-0008 Proposed through the feedback window, then Accept and implement before or as the next production release — not rush a half-specified pipeline under the aarch64 cut.

[lawrennd]

Follow-up: also tagging @ekalosak (Eric Kalosa-Kenyon) — recent release/CI history includes v1.10.0 (#908), deploy↔devel merges, Travis/macOS fixes (#904), and Python 3.9 builds (#888). Sorry for the miss on the original list; your take on CIP-0008 would be especially useful.

[lawrennd]

Update 2026-10-24. 1.15.0 used the existing GitHub Release trigger (same path as 1.14.x). Publishing v1.15.0 queued three Test Python Lib runs, which matches on: release without types: — one of the CIP-0008 bugs to fix.

CIP-0008 said to stay Proposed through that production cut. The cut is done. There were no replies on the open questions besides the extra maintainer ping, so we will Accept CIP-0008 and implement:

1. Rename default devel → main (no main branch exists today; deploy last moved in July 2024).
2. Drop deploy from CI triggers.
3. on.release.types: [published] only.
4. Wheel dry-run via workflow_dispatch (no twine).
5. Trusted Publishing plus tag == poetry version -s (lynguine’s shape; keep GPy’s OS/Python/Cython wheel matrix).

Comments still welcome if anyone relies on devel/deploy. Implementation will wait until 1.15.0’s deploy job finishes so we are not rewriting the pipeline under a live upload.

[sronix]

I recently set up something similar for a project of mine (publish from a version tag, check the tag against the package version, OIDC to PyPI), so here are a few notes on the Trusted Publishing and environment part.

The CIP says the deploy job still uses twine with TWINE_*, but those env lines seem to have come out in August (d6bbf9f). The 1.15.0 deploy log has twine fetching the OIDC token itself ("Got OIDC token for audience pypi"): https://github.com/SheffieldML/GPy/actions/runs/40130966985/job/120984126014. Since 6.1, twine tries Trusted Publishing when no token is configured and the job has id-token: write, so GPy looks to be on Trusted Publishing already. That makes the move to pypa/gh-action-pypi-publish optional. The main thing it adds is PEP 740 attestations, which it generates by default.

There's already a release environment in the repo with no protection rules, and the deploy job doesn't use it yet. If you add environment: release to the deploy job and give the environment a deployment rule for v* tags, only a v* tag ref can reach the upload. A published Release runs on its tag ref, so releases still go through. That limits which ref can publish, but not who: anyone with write access can publish a Release. If you want an approval step before each upload, required reviewers on the environment add one (the environment's self-review and admin bypass settings are worth a look too).

On PyPI, the trusted publisher can name the environment too, and then PyPI refuses tokens from runs outside it. I'd add the environment to the workflow first and to PyPI second, since the other order breaks the next upload.

[lawrennd]

Thanks for spotting this!

As you say the TWINE_* lines did come out in d6bbf9f. CIP-0008’s “still uses twine + TWINE_*” is stale; 1.15.0 already published via twine 6.1 OIDC with id-token: write. We will keep twine upload rather than move to pypa/gh-action-pypi-publish unless we specifically want PEP 740 attestations.

The unused release environment is the remaining CIP-0008 slice. We will attach environment: release to the deploy job (and a v* tag rule) before naming that environment on the PyPI trusted publisher, so we do not block the next upload.

I'll correct the CIP text to say we are already on Trusted Publishing via twine.

[lawrennd]

Thanks @sronix — that reading matches the tree.

The TWINE_* lines did come out in d6bbf9f. CIP-0008’s “still uses twine + TWINE_*” was stale; 1.15.0 already published via twine 6.1 OIDC with id-token: write. We will keep twine upload rather than move to pypa/gh-action-pypi-publish unless we specifically want PEP 740 attestations.

The unused release environment is the remaining CIP-0008 slice. We will attach environment: release to the deploy job (and a v* tag rule) before naming that environment on the PyPI trusted publisher, so we do not block the next upload.

The CIP text is corrected in #1258 (9dd860ae).