ci(release): gate the tag point — a tag behind main silently drops work (#366)

[avrabe]

Closes #366.

What broke, twice

tag	what it excluded	margin
v0.31.0	9557dd0 — fix(parser): reject C-style radix literals (0x/0b/0o) instead of silently parsing to 0	5m06s
v0.33.0	two commits	—

Both with check-versions green.

Why check-versions could never have caught it

Not an oversight — a category error. That job measures the version window: it asserts the tag matches Cargo.toml, and Cargo.toml reads X.Y.Z from the bump commit until the next bump. That window spans both the commits a tag included and the ones it silently left out. The invariant is circular with respect to the question being asked.

The gate I designed first was also wrong

Recorded in #366 so it isn't re-proposed. Asking "which release does this commit belong to" makes the verdict a function of what has merged since — a gate whose answer drifts as unrelated commits land is not a gate. Filtering by tagger date doesn't rescue it either: under that design the stragglers predate the tagger date anyway, because the tag itself was held.

What it actually asks

Is anything that was already finished being left out, and did you say so?

Given tag vX.Y.Z at commit C with tag date T:

1. C not an ancestor of origin/main → skip (release-branch tag)
2. enumerate C..origin/main, restricted to --before=T
3. empty → pass
4. non-empty and the tag message states a tag point → pass, listing them
5. non-empty and it does not → fail, listing them

Step 2's --before=T is doing two jobs. Race-proof: a merge landing seconds after the tag push postdates T and cannot fail the job. Idempotent: both inputs are immutable, so a re-run months later returns the same verdict.

Evidence

Backtest — all 42 v0.* tags, running the shipped script, not a transcription of it:

39 PASS   1 PASS-ACK (v0.34.0)   2 FAIL (v0.31.0, v0.33.0)

The two fails are exactly the two independently-confirmed bad tags. No false positives across 40 good tags. v0.34.0 passes via rule 4 because its message already says "Tag point is the bump commit ff470a2 … deliberately held after the bump merged, under the batching policy" — the acknowledgement convention already exists in how these tags are written; the gate enforces a habit that was optional and got skipped exactly when it mattered.

The real failure output names the cause directly:

::error::v0.31.0 was cut at 98bc574c (2026-07-22T21:05:22+02:00), but 1 commit(s) were
::error::already on main before that and are not in the release:
::error::  9557dd0 2026-07-22 21:00:06 +0200 fix(parser): reject C-style radix literals ... (#343)

Mutation test — is rule 4 load-bearing, or does v0.34.0 pass for some other reason? Two scratch tags on the identical commit ff470a2, yielding the identical 8-commit excluded set, differing only in whether the message contains "tag point":

tag message	verdict
"…deliberately held after the bump merged, under the batching policy."	FAIL (exit 1)
"Tag point is the bump commit ff470a2; held under the batching policy."	PASS (acknowledged)

Distinct inputs → distinct verdicts. (Both scratch tags deleted; git tag -l 'zz-*' → 0.)

Two deliberate structural choices

It lives in tools/ci/check-tag-point.sh, not inline in the YAML. A backtest of shell transcribed into a workflow proves the transcription works. Putting it in a file means the 42-tag backtest and the mutation test exercise the artifact that actually ships — the same lesson as the withdrawn #327 certificate, where an oracle that couldn't be run against distinct inputs certified nothing.

It gates create-release, not the builds. A red verdict stops everything outward-facing (publish-vsix already needs: create-release) while the maintainer still gets full build feedback to act on.

What this does not do

Stated in the script header rather than left to be discovered:

• It cannot verify the acknowledgement is true. It greps for a stated tag point; "tag point: whatever" passes. A speed bump against autopilot, not a proof. Its value is forcing the tagger to look at the list — precisely what did not happen twice.
• It cannot catch tagging a commit that is too new. Different failure.
• It cannot catch work merged after the tag. Nothing at tag time can, and step 2 deliberately stops trying.
• It assumes tag and committer dates come from comparably-set clocks. They do here. A skewed clock weakens step 2 in the permissive direction — it under-reports, never over-reports.

Scope

CI plumbing, no rivet artifact — matching the #353/#363/#364 precedent; this repo carries no REQ-CI-* ids. Flagged so the omission reads as a choice, not a miss. It also keeps this PR clear of artifacts/requirements.yaml, which #368/#373/#374 are all currently touching.

🤖 Generated with Claude Code

[github-actions]

Rivet verification gate

✅ 20/20 passed

	count
Passed	20
Failed	0
Skipped (no steps)	0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

_{Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[avrabe]

Clean-room audit: this gate could be switched off silently, three ways

Same cold subagent, same instruction — refute, don't confirm. It confirmed A1
(the 42-tag backtest), A2 (v0.31.0 excluded 9557dd0, merged 5m16s before
the tag — the header said 5m06s) and A3. It refuted the rest, and one finding is
severe enough that merging this as written would have shipped a gate that
disarms on the sloppier path.

1. A lightweight tag made the gate vacuous — unconditional PASS

creatordate on a lightweight tag is the tagged commit's own committer
date, so --before=$T asks for commits descended from C that predate C —
on a linear history, always empty. Demonstrated on this repo, same commit
945799a tagged both ways:

annotated   -> ::error:: ... 13 commit(s) were already on main   rc=1
lightweight -> PASS: nothing predating the tag was left out      rc=0

Compounding: %(contents) on a lightweight tag returns the commit message,
so the acknowledgement could be satisfied by text the tagger never wrote.

The trigger is git tag v0.36.0 instead of git tag -s v0.36.0. The gate
switched itself off on exactly the mistake that most needs gating. My header
called this "a slightly weaker check" — it was no check. Now rejected outright.

2. The acknowledgement disarmed itself

Rule 4 grepped the whole tag message for tag[ -]point. The commit that adds
this gate has the subject ci(release): gate the tag point — …, and release
notes here list commit subjects verbatim (v0.35.0's do). The first release
whose notes quoted that subject would auto-acknowledge. Reproduced: a
realistic CONTENTS block passed with zero acknowledgement.

Now anchored to the start of a line — a quoted subject sits mid-line after a
short hash; an acknowledgement is something the tagger began a line with. I
tried trailer-only (Tag-Point:) first and rejected it: it fails v0.34.0, whose
"Tag point is the bump commit ff470a2" is specific and correct. A gate that
rejects real acknowledgements teaches people to route around it.

3. Not idempotent — and it over-reported, contradicting my soundness claim

--before filters committer date (when a commit was written); the question
is when it landed on main. A branch dated the 3rd, merged the 10th, injects
commits "dated before T" into a range they weren't reachable from at tag time.
In a scratch repo — byte-identical tag object, only main moved:

RUN 1 (at tag time)      PASS   rc=0
RUN 2 (after the merge)  ::error:: 1 commit(s) were ...   rc=1

That is the verdict-drift #366 explicitly rejected in its first design,
reintroduced one level down. And it's an over-report — a false accusation
against a tagger who did tag the tip of main — refuting the header's claim to
err "in the PERMISSIVE direction". That sentence is deleted rather than softened.

Fix: --first-parent. The spine holds exactly the squash and merge commits,
whose committer dates are their landing times, so --before becomes exact
instead of approximate. A late-merged branch contributes its merge commit (dated
after T, correctly excluded), not its old constituents. Same scratch repo now
gives PASS / PASS.

Detection is untouched: v0.31.0's missed commit 9557dd0 is a squash commit on
the spine. This repo squash-merges, which made the defect latent — but main
carries 91 merge commits, so latent is not impossible.

Verification

Backtest over all 42 tags, unchanged end-to-end: 39 PASS, 1 acknowledged
(v0.34.0), 2 FAIL (v0.31.0, v0.33.0) — the two real misses #366 was filed for.
All three counterexamples above now exit 1 (measured directly, not through a
pipe). Scratch tags cleaned up; git tag -l 'zz-fix-*' → 0.

The holes closed without moving a single verdict on real history.

🤖 Generated with Claude Code