Skip to content

Publish prod manifests through an immutable staging reference so latest never names unsigned bytes #2849

Description

@devantler

🤖 Generated by the Agentic Engineer

Evidence

ghcr.io/devantler-tech/platform/manifests:latest is moved by ksail workload push, and signing does
not begin until three steps later. Measured on main (2026-07-28), step positions in
.github/actions/deploy-prod/action.yml:

# Step
9 📦 Push manifests to GHCR — latest now points at the new bytes
12 🖋️ Sign OCI manifest with cosign
14 🪪 Attest SBOM
15 🪪 Attest build provenance
18 🔁 Trigger Flux reconciliation

There is no staging reference: the mutable tag is the publication target, so the artifact is reachable
under the name production tracks for the whole span between steps 9 and 15.

Audience and problem

The cluster's root Flux source polls that tag on a 1-minute interval and delivers the entire platform —
every controller, tenant binding and policy. Today spec.verify on that source is null, so the window is
latent. It becomes load-bearing the moment root verification is switched on (#2627's other child): Flux
can resolve bytes whose signature or attestations do not exist yet.

Hypothesis

Publishing to an immutable reference, completing the evidence against it, and only then repointing
latest closes the window without changing what production ultimately runs.

First step is measurement, not a guess

Open question to answer first: whether ksail workload push can target a reference other than
latest. If it can, this is a deploy-path change. If it cannot, the capability belongs in KSail and this
issue depends on that work. Nothing else here should be designed before that is settled.

Acceptance criteria

  • latest is only repointed once the signature and both attestations exist for that exact digest.
  • A failed push, sign, or attest leaves the previously verified artifact active under latest.
  • The DR rebuild path publishes through the same transaction (it writes the same tag).
  • A static test pins the transaction, in the shape of scripts/validate-publication-order (test(ci): pin the prod publication ordering so evidence precedes release #2848).

Rough size: M — the transaction itself is small; the KSail capability question is what decides whether
this is one repo or two.

Part of #2627

Metadata

Metadata

Assignees

Labels

Projects

Status
✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions