ci(workers): publish SHA-tagged workers image alongside :latest-workers#238
Merged
therealbrad merged 1 commit intomainfrom Apr 23, 2026
Merged
ci(workers): publish SHA-tagged workers image alongside :latest-workers#238therealbrad merged 1 commit intomainfrom
therealbrad merged 1 commit intomainfrom
Conversation
Every workers image build now also publishes
`ghcr.io/testplanit/testplanit:workers-sha-<short-sha>` (8-char git SHA).
The existing :latest-workers alias is preserved for convenience, but
the multitenant-workers Deployment should pin to the SHA tag so
`kubectl rollout undo` has real history to roll back to — during the
2026-04-22 incident, rollback was impossible because every Deployment
revision pointed at the same floating :latest-workers tag, so the
broken image had already overwritten the good one in the registry.
Changes in .github/workflows/release.yml:
- build-amd64 / build-arm64 (tag-push): add
--set workers.tags=...:workers-sha-${SHORT_SHA}-<arch>
- docker-manual-amd64 / docker-manual-arm64: same
- merge-manifests / docker-manual-merge: create multi-arch manifest
ghcr.io/testplanit/testplanit:workers-sha-${SHORT_SHA}
Companion work (NOT in this PR, since the manifest lives elsewhere):
the multitenant-workers Deployment in the private-ops repo must be
updated to reference :workers-sha-<short-sha> instead of :latest-workers.
Until that PR lands, this repo publishes the SHA tag but nothing
consumes it — safe no-op in that intermediate state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
|
🎉 This PR is included in version 0.22.8 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Publishes a SHA-pinned workers image tag on every build so the k8s Deployment can stop pointing at the floating
:latest-workerstag.Background. On 2026-04-22, rollback of the multitenant-workers crashloop was effectively impossible: every revision in
kubectl rollout historypointed at the same tag (:latest-workers), and the broken image had already overwritten the good one in GHCR. "Undo" had nowhere to roll back to.What this PR does — in
.github/workflows/release.yml:SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-8)inside each build job.ghcr.io/testplanit/testplanit:workers-sha-<short-sha>-<arch>ghcr.io/testplanit/testplanit:workers-sha-<short-sha>(manifest list over amd64 + arm64)Nothing is removed.
:latest-workersand:${VERSION}-workersare still published and:latest-workersis still retagged by the existing "is this the newest semver" logic. This PR is purely additive.What is NOT in this PR
The k8s Deployment manifest that references
:latest-workerslives in the private-ops repo, not here (see the comment block intestplanit/k8s/multitenant-workers-rbac.yaml— it describes this split). A companion PR in private-ops is needed to change the Deployment'simage:to:workers-sha-<short-sha>, parameterized per deploy.Intermediate state while that PR is pending: this repo publishes the SHA tag but nothing consumes it yet. That's safe — the existing
:latest-workerspath is unchanged, so no behavior differs until ops flips the Deployment.Related Issue
Follow-up to hotfix c804cac (workers crashloop 2026-04-22 16:30 UTC). Third of three post-mortem follow-ups:
auditContextWrappers.tsso workers don't transitively import Next.js (root-cause fix).Type of Change
Technically a CI/ops-infrastructure fix. No runtime code changes.
How Has This Been Tested?
The workflow file itself was validated:
js-yamlload, six jobs present).SHORT_SHAand add aworkers-sha-${SHORT_SHA}-<arch>tag to the bake--set.:workers-sha-${SHORT_SHA}.The first real tag-push after this merges will publish the new tag as a side effect — that's the soonest a live run happens. The changes are additive and the tag-creation logic mirrors the established shape of the existing version/workers tags, so risk of breaking the current release path is low.
Test Configuration:
Checklist
Additional Notes
Tag format choice. Short 8-char SHA (e.g.,
workers-sha-c804cace) matches the example in the post-mortem and the convention elsewhere in the project. Full 40-char SHAs are unambiguous but unwieldy as image tags.Why workers only, not production. The 2026-04-22 incident was workers-specific and the user's post-mortem scoped this PR to workers. The production image has the same failure mode in principle (
:latestis also floating), but extending the SHA-tag convention to production is a separate decision with its own deploy-flow implications. Happy to open a follow-up if ops wants it.Rebase note. #237 also touches
release.yml(adding smoke-test steps to the same build jobs). The two PRs are logically independent; whichever merges first, the second needs a trivial rebase — edits are in adjacent-but-not-overlapping blocks.