Rehearse every publish path on workflow_dispatch - #673
Merged
Conversation
Dispatching the six release workflows was supposed to be a full rehearsal
of a release, but only ruby, typescript and kotlin actually exercised
their publish path. The Go tag job carried a job-level
`if: github.event_name == 'push'` and the Swift verification steps were
step-gated the same way, so on a manual run they were simply skipped: a
break in either would first surface during the irreversible tag push.
Go could not just be ungated. The job derives its tag from
`TAG="${GITHUB_REF#refs/tags/}"`, and on a dispatch GITHUB_REF is
`refs/heads/<branch>`, so that expansion yields the literal string
`refs/heads/main`. The rehearsal now reads the version from
go/pkg/basecamp/version.go, the same way release-ruby reads version.rb
and release-typescript reads package.json, computes the `go/vX.Y.Z` tag,
and reports whether it would create it, leave it alone, or move it.
The dry run lives in its own job rather than in the tagging job so that
it holds `contents: read`; only the push path keeps `contents: write`.
`git tag` and `git push` remain gated on the push event and the push
path is otherwise untouched.
Swift distributes through SPM, so the release artifact is the git tag
itself. Its rehearsal reads the version from BasecampConfig.swift,
checks that the package manifest resolves, reports the tag SPM will
resolve from, and builds and tests as before.
Kotlin and TypeScript gain a "Would publish" line so all six workflows
now print the same three things on a dispatch: the version read from
that SDK's own source of truth, exactly what would be published, and an
unmistakable dry-run notice.
Contributor
Sensitive Change Detection (shadow mode)This PR modifies control-plane files:
|
The dispatch rehearsal for python could not run at all: its only publishing job carries `environment: release-pypi`, whose deployment branch policy is `v*`-only, so a manual run from a branch is refused before any step executes. Ruby and typescript have the same shape and would be refused the same way from any ref that is not a release tag. Widening the policy would buy a rehearsal by weakening the thing that makes the environment worth having. Split each of those three workflows instead. An unprotected build job reads the version from that SDK's own source of truth, builds the artifact, prints what it would publish, and uploads the result. An environment-gated publish job downloads that artifact and publishes exactly those bytes. The environment now gates only real publishes, the rehearsal needs no deployment approval and works from any ref, and the bytes that ship are the bytes that were built and validated rather than a rebuild that nobody looked at. For typescript that means packing a tarball rather than letting `npm publish` rebuild through prepublishOnly, and publishing the tarball by path. The build is now explicit because `npm pack` does not run prepublishOnly. Kotlin gets the same least-privilege treatment in the shape its publisher allows. Gradle's publish task builds from the project rather than from a prepared file, so there is no artifact to hand over; the rehearsal moves into its own job so that it no longer runs in a job holding `packages: write`.
The npm dry run now runs the exact command the publish job runs on a tag, flags and all, with --dry-run added. npm resolves the manifest out of the tarball, lists the file set it would upload and negotiates with the registry without writing, which is what proves that publishing a packed tarball behaves like publishing a directory. A probe run confirmed --provenance is accepted alongside --dry-run on npm 11.16.0 and Node 24.18.0 even in a job with no id-token permission, so the invocation is exercised whole; only the attestation, which is minted from the OIDC token at publish time, is left unproven. gem push has no dry run, so the gem is verified as an artifact instead. The gemspec collects its file list from git ls-files, so a checkout that did not produce one would build a well-formed gem containing no library — something RubyGems accepts and nobody can require. The check reads the packaged spec back and asserts name, version and the presence of lib/. Python gains twine check, the same metadata and long-description validation PyPI applies at upload. Not --strict: the package ships no long_description at all today, so --strict would fail every release over a pyproject.toml defect rather than a workflow one.
jeremy
added a commit
that referenced
this pull request
Aug 5, 2026
#673 and #675 landed after the last measurement, so the guide's baseline moves from 931c36a to 70d576b. The counting method changes too, because the one committed here was wrong. `git log v0.12.0..HEAD | wc -l` counts COMMITS, which equals PRs only while every commit is a squash merge. The release-prep commit is pushed straight to main and is not a PR, so from the moment the version is bumped that count runs one high — it reported 59 where 58 PRs had merged. That is the third way this number has been wrong, each differently: a lexicographic string compare on timestamps, then a temporal compare that credited #556 to this release (its squash commit IS the commit v0.12.0 tags), and now a commit count meeting its first non-PR commit. The rule that survives all three is the definition itself: reachable from HEAD and not reachable from v0.12.0, applied to the merge commits of PRs rather than to commits. Both derivations in the guide now use it, and both were run to confirm they reproduce the numbers printed beside them. Committed directly rather than as a pull request, on purpose: a PR carrying this number would increment the number it states. #642 shipped wrong for exactly that reason — a document inside the release cannot count itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dispatching the six
release-*.ymlworkflows is meant to be a full rehearsal of a release. Rehearsing them frommainat7085ba3b1(the 0.13.0 bump) showed only three of the six exercising a publish path, and one that could not be dispatched at all.A skipped job is not a rehearsal. If the Go tagging step or the Swift version check were broken, nothing would say so until the tag push — and that push is irreversible, so the first evidence of a broken publish path would arrive at the worst possible moment.
What was wrong, per SDK
tagjob carried a job-levelif: github.event_name == 'push'and was skipped entirely.environment: release-pypi, whose deployment branch policy isv*-only. A manual run from a branch is refused before any step executes, so the wheel was never even built in rehearsal.maintoday, but the environment is on the job that does the building, so the rehearsal is hostage to a deployment approval it does not need.packages: write.The
GITHUB_REFdefect in release-goThe Go tag job could not simply have its job-level
ifremoved. It derives the tag it pushes fromTAG="${GITHUB_REF#refs/tags/}"On a
workflow_dispatchGITHUB_REFisrefs/heads/<branch>, notrefs/tags/v*, so the prefix strip is a no-op andTAGbecomes the literal stringrefs/heads/main. Ungating it naively would have "rehearsed" creating a tag namedgo/refs/heads/main.The dry run therefore reads the version from
go/pkg/basecamp/version.go, the same way release-ruby readsversion.rband release-typescript readspackage.json. It computesgo/vX.Y.Z, checks the remote withgit ls-remote, and reports which of the three real branches it would take: create, leave alone, or force-move.Build outside the environment, publish inside it
For python, ruby and typescript the fix is a split rather than a policy change. No deployment branch policy is widened;
release-pypistaysv*-only.environment:,contents: read— runs on both events. It reads the version from that SDK's own source of truth, builds the artifact, prints what it would publish, and uploads the result withactions/upload-artifact.needs: build, job-levelif: github.event_name == 'push'— downloads that artifact and publishes exactly those bytes. It does not rebuild.Two things fall out. The environment now gates only real publishes, so a rehearsal needs no deployment approval and works from any ref, including a fork. And what ships is what was built and validated, not a rebuild nobody looked at.
For typescript that means packing a tarball instead of letting
npm publishrebuild throughprepublishOnly, then publishing the tarball by path.npm packdoes not runprepublishOnly, hence the now-explicitnpm run build. This is the one place where the tag-push invocation itself changes shape (npm publish <tarball>rather thannpm publish <dir>), so the rehearsal runs that exact invocation — see below.Kotlin is the exception, deliberately. Gradle's publish task builds from the project rather than from a prepared file, so there is no artifact to hand over and no way to give kotlin the build-once property without hand-rolling Maven upload, which would be a larger and riskier change than the gap it closes. Kotlin is therefore the one workflow where the bytes published on a tag are rebuilt by the publish job rather than handed over from the build that was validated. It still gets the least-privilege half of the treatment: its rehearsal moved into its own job so it no longer runs under
packages: write.Rehearsing the real invocation, not an approximation
Each build job now runs whatever native dry run or artifact validation its packaging tool actually offers, so the rehearsal exercises the publish path rather than asserting it should work.
npm —
npm publish "$TARBALL" --access public --tag "$NPM_TAG" --dry-run --provenance: the publish job's exact command with--dry-runadded. npm resolves the manifest out of the tarball, enumerates the file set it would upload, and negotiates with the registry without writing. A probe run confirmed--provenanceis accepted alongside--dry-runon npm 11.16.0 / Node 24.18.0 in a job holding noid-token: write, so the whole invocation is exercised. The attestation itself is minted from the OIDC token at publish time and is the one part a dry run cannot prove.RubyGems —
gem pushhas no dry run, so the artifact is verified instead. The gemspec collects its file list fromgit ls-files; a checkout that did not produce one would build a well-formed gem containing no library, which RubyGems accepts and nobody can require. The check reads the packaged spec back out of the.gemand asserts name, version, and the presence oflib/.PyPI —
twine check dist/*, the same metadata and long-description validation PyPI applies at upload. Deliberately not--strict: see the finding below.Finding: the Python package has no long description
twine check --strictfailed the first probe run:python/README.mdexists (33 KB) butpython/pyproject.tomldeclares noreadme, so the sdist and wheel carry no long description and the PyPI project page forbasecamp-sdkrenders empty. PyPI accepts this, so it is not an upload blocker and--strictwould fail every release over it. That is a one-linepyproject.tomlfix, not a workflow fix, and it changes the metadata of the artifact this release ships — so it is reported rather than smuggled in here. The workflow runs plaintwine check, which passes with those warnings visible in the log.Job graph after
environment:testcontents: readtag-dry-runcontents: readtagcontents: writetest(+ dry-run steps)contents: readtestcontents: readpublish-dry-runcontents: readpublishcontents: read,packages: writebuildcontents: readpublishrelease-rubygemscontents: read,id-token: writebuildcontents: readpublishrelease-npmcontents: read,id-token: writebuildcontents: readpublishrelease-pypicontents: read,id-token: write,attestations: writeEvery job that runs on a dispatch holds
contents: readand nothing else. Every write permission and everyenvironment:now sits on a job that only runs on a tag push.git tagandgit pushremain gated ongithub.event_name == 'push'.The
Verify tag is on mainguard moved with the checkout into each build job. It still runs on push and still blocks the publish job throughneeds:.The publish job gets this run's artifact, not a rebuild and not a stale one
Three things pin this down.
needs: buildmakes the publish job run only after the build job in this run succeeded.actions/download-artifactdefaultsrun-idto${{ github.run_id }}andrepositoryto${{ github.repository }}— its own action.yml says it "will attempt to download artifacts from the current repository and the current workflow run" when nogithub-tokenis given, and no token is given here — so it cannot reach a previous run's artifact. And the publish job has no checkout and no build tooling: there are no sources present to rebuild from even if a step tried. The same wiring holds for ruby (rubygem) and typescript (npm-tarball).Verification
make lint-actions(actionlint + zizmor) green:No findings to report. Good job! (17 ignored, 34 suppressed), real exit code 0.All six dispatched against this branch ref at
42c11c848. Every previously-skipped job or step executed, every workflow read0.13.0from its own source of truth, and every publish job was skipped:Python is the acid test — it built and rehearsed from a branch with
release-pypistillv*-only:Go, previously skipped entirely:
Swift, previously skipped:
Ruby —
gem pushhas no dry run, so the built artifact is read back:TypeScript — the publish job's exact invocation, dry:
Python —
twine checkagainst the artifact that would be uploaded:Kotlin:
Nothing was mutated. After all six runs:
The only side effect is the build artifacts attached to the runs themselves, at 7-day retention.
No action SHA pins were changed;
actions/upload-artifactreuses the pin already in this repo andactions/download-artifactis pinned to v8.0.1, which shares@actions/artifact^6.2 with it.release-github.ymlis untouched.