fix(ci): make Release single-flight and idempotent to end double-trigger 422s#491
Merged
Conversation
…ger 422s The Release workflow is triggered redundantly on purpose: a candidate tag push emits a real event while orchestrate, promote, and auto-promote also dispatch it explicitly, because API-created and CI-skipped tags do not emit push or release events (#86). A pushed tag then starts two runs at once, the loser hits GoReleaser 422 already_exists, and each cycle leaves a red run and a stray draft. Add a constant single-flight concurrency group so duplicate runs serialise instead of racing, and an idempotency guard that skips the build and finishes green when the tag is already published with its checksum manifest. A belt after GoReleaser treats a 422 as success only when the tag is already published. The decision core lives in a script and is unit tested. Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
joshua-temple
added a commit
that referenced
this pull request
Jul 6, 2026
…eneration The tag-only/non-triggering-tag-create behavior is cascade's own release plumbing (cascade self-publishes via GoReleaser), not a feature every downstream manifest should see. Move it off the public manifest surface. - remove the public release.tag_only manifest field entirely: config schema, validation, and docs are back to plain generation; downstream manifest generation is byte-identical to before this change - add an own-repo generation mode instead, mirroring the reconcile companion's WithOwnRepo: NewGenerator gains a variadic GeneratorOption tail and WithOwnRepoRelease(), which is reachable only from a new --own-repo flag on generate-workflow and verify (never from config) - the manage-release composite action template only adds the tag_only input (and its env/CLI forwarding) when generated in own-repo mode; plain generation carries no trace of it - regenerate cascade's own orchestrate.yaml and manage-release action with --own-repo; leave promote.yaml untouched (cascade's real final publish is the hand-authored auto-promote.yaml; promote.yaml is inert for cascade) - wire --own-repo into the PR workflow-drift check and its regenerate hints so cascade's own-repo generated files are drift-locked correctly - rework e2e scenario 46 to exercise generate-workflow --own-repo instead of a manifest field, and add scenario 47 as the paired plain-generation regression guard - keep the release.go/command.go runtime mechanics (tag-only create() short-circuit, --tag-only CLI flag, non-triggering token, #491 backstop) unchanged; only the scoping vehicle changed Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
joshua-temple
added a commit
that referenced
this pull request
Jul 6, 2026
…496) * fix(release): one Release run per tag and no orphan draft on rc cut Cascade's own rc cut and final publish each drove two Release runs per tag (the state-token tag push fired push: tags while orchestrate/promote also dispatched Release) and the rc cut left an orphaned draft that GoReleaser duplicated. Both are now single-fire, mirroring the auto-promote pattern. - add release.tag_only (schema, validation, manage-release --tag-only): cut the tag only, skip the draft-release POST, and create the tag with the non-triggering GITHUB_TOKEN so the tag push does not fire the release workflow alongside the explicit dispatch - orchestrate rc cut and promote publish emit tag_only + GITHUB_TOKEN for the tag-create step in tag-only mode, keeping the explicit dispatch on the PAT so the Release run still cascades to fleet-e2e via workflow_run - scope tag_only to cascade's own manifest; downstream generated output is byte-identical (regression-guarded), so a repo without GoReleaser keeps its draft-then-publish flow - keep push: tags for the hotfix and human ad-hoc tag paths; dispatch is canonical for machine paths - retain the release concurrency group, release-idempotency.sh, and benign-422 belt as a defensive backstop - regenerate cascade's orchestrate/promote/manage-release; set tag_only in the manifest; e2e scenario 46; unit and regression tests; docs Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com> * fix(release): scope tag-only release plumbing to cascade's own-repo generation The tag-only/non-triggering-tag-create behavior is cascade's own release plumbing (cascade self-publishes via GoReleaser), not a feature every downstream manifest should see. Move it off the public manifest surface. - remove the public release.tag_only manifest field entirely: config schema, validation, and docs are back to plain generation; downstream manifest generation is byte-identical to before this change - add an own-repo generation mode instead, mirroring the reconcile companion's WithOwnRepo: NewGenerator gains a variadic GeneratorOption tail and WithOwnRepoRelease(), which is reachable only from a new --own-repo flag on generate-workflow and verify (never from config) - the manage-release composite action template only adds the tag_only input (and its env/CLI forwarding) when generated in own-repo mode; plain generation carries no trace of it - regenerate cascade's own orchestrate.yaml and manage-release action with --own-repo; leave promote.yaml untouched (cascade's real final publish is the hand-authored auto-promote.yaml; promote.yaml is inert for cascade) - wire --own-repo into the PR workflow-drift check and its regenerate hints so cascade's own-repo generated files are drift-locked correctly - rework e2e scenario 46 to exercise generate-workflow --own-repo instead of a manifest field, and add scenario 47 as the paired plain-generation regression guard - keep the release.go/command.go runtime mechanics (tag-only create() short-circuit, --tag-only CLI flag, non-triggering token, #491 backstop) unchanged; only the scoping vehicle changed Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com> --------- Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Release workflow is triggerable three ways (
push: tags: v*,release:, andworkflow_dispatch) with noconcurrencygroup. The redundancy is intentional (#86): orchestrate, promote, and auto-promote each dispatch Release explicitly because API-created and CI-skipped tags do not reliably emit push or release events.When a candidate tag is cut by a real
git push(which does emit a push event) and the dispatcher also fires its explicit dispatch, both trigger Release at the same instant. Two parallel runs race GoReleaser; the loser hits422 already_exists, leaving a red Release run and a stray draft every cycle. This happened for v0.9.1-rc.1 (runs 28800047479 push/success and 28800046933 dispatch/failure, identical createdAt).Fix
Make the duplicate harmless rather than removing a trigger (the fallback is still required for the inverse failure mode where the push event is suppressed):
checksums.txtmanifest for the tag. A belt after GoReleaser (run undercontinue-on-error) converts a GoReleaser failure into success only when the tag is already published, and fails the job on any other error. The decision core is.github/scripts/release-idempotency.sh, unit tested ininternal/release/release_idempotency_test.go(existing-release-with-checksums to skip, draft/no-checksums/absent to build, and the 422-tolerance predicate).concurrency: { group: release, cancel-in-progress: false }, a constant group rather than one keyed on the tag, enforces the hard invariant that no two releases are ever in flight together and serialises the double-trigger.Cancel-pending safety analysis
A shared group with
cancel-in-progress: falsestill cancels all-but-the-latest pending run. Trace for a legitimate final publish:GITHUB_TOKEN, whose pushes do not trigger workflows, then dispatches Release once. It is also temporally far from the rc build runs (a full fleet gate, tens of minutes, sits between them), so it enters thereleasegroup alone and is not pending behind anything in the normal flow.Conclusion: the global group is safe; a legitimate final publish is either not pending, or is recoverable by the idempotent, watched, redispatchable path. No design change away from the global group is needed.
Per-caller trace
releasegroup; publishes or idem-skips if already published.release:event fallback: same guards apply; a duplicate no-ops.Verification
go build ./...,go test ./...(2448 passed),golangci-lint run ./...(no issues),go vetclean.actionlint .github/workflows/release.yamlclean;shellcheckclean on the new script.release-orchestration.md, new "Release runs once per tag" section). Local Node is v20.20.2 (below 22.12); CI builds the docs site. The addition is plain prose and will not break the build.Residual risk
The 422 belt runs GoReleaser under
continue-on-error, so a GoReleaser failure is only surfaced by the follow-up confirm step. That step fails the job on any failure other than an already-published tag, so real failures still red the run.