Prove every marker the release box classifies by, or the coverage test goes red

[Antawari]

Stacked on #258. Base is catrina/gates-r3-bump-the-kit-pin, so this diff is this rung only. Merge #258 first; GitHub will retarget this to main.

CONTRACT

When a pip step in the release-gate box fails, classify_pip_failure (tests/e2e/scripts/e2e-runner.sh) decides whether the verdict blames the box's network (box_network_unreachable, exit 11) or the artifact under test (artifact_install_failed, exit 8). Both are a FAIL. Only the claim differs — and the claim is load-bearing, because docs/box-operator.md and docs/release-gates.md teach operators to read the artifact_* family as "the box worked and the artifact did not."

On 2026-07-27 that reason was emitted twice for a run whose link had collapsed to 13 KiB/s and which never installed, imported or executed the wheel. The same wheel installs into a clean venv in 1.4 seconds. A verdict naming the artifact when the artifact was never reached is a false accusation against a released wheel, and it points the next operator at the wrong repository.

WORLD — the defect is vacuity in the marker lists, and it is worse than expected

I was briefed that the classifier was "proved only against a dead link, never the degraded one." That is partly stale and this PR does not repeat it — test_e2e_runner_install_resilience.py already proved DNS failure, the read-timeout signature of the 2026-07-27 incident itself, connection-refused, four artifact shapes, and a mixed corrupt-wheel-on-a-dead-link rod.

Measured properly, the real hole is worse:

list	alternatives	appeared in any test	load-bearing
transport_markers	17	6	0
artifact_markers	10	4	2

Zero of seventeen transport alternatives were individually proved. Each of the three network fixtures matched two or three alternatives at once, so deleting any single transport alternative changed no test result. LOG_ARTIFACT_BUILD_CRASH likewise carries both build markers, so neither is proved.

A pattern list nobody proved matches is a gate selecting by a value it never grades. A typo, a case change, or an upstream pip rewording in any transport alternative silently reclassifies a dead link as a broken wheel — the exact failure this function was written to prevent.

What this adds

Every alternative in both lists now has captured pip output that matches it and nothing else, so deleting that alternative flips a real verdict. The degraded shapes the incident class is actually about had no coverage at all and are covered now:

condition	marker
body truncated mid-download at 13 KiB/s	IncompleteRead
chunked response whose final chunk never arrives	ChunkedEncodingError
middlebox RemoteDisconnected mid-download	Connection aborted
errno-104 reset with no urllib3 class in the line	Connection reset by peer
connect-phase stall (says "timed out", not "Read timed out")	ConnectTimeoutError
all 8 retries spent against a 503-ing index	MaxRetryError
captive portal / corporate MITM presenting its own CA	SSLError
portal answering the index with 403 — pip skips rather than raising, so the log otherwise looks like a genuinely missing package	Could not fetch URL
name resolution failing a second way (gaierror -2)	Name or service not known
box with no default route	Network is unreachable

Every artifact alternative is graded THROUGH a degrading link, because an artifact-only log classifies as artifact via the default branch and therefore proves nothing about the ordering rule. Four mixed shapes are named, including one where the unpack failure prints before the transport noise, so the rule cannot be passing by reading position.

The part that keeps it true — the non-vacuity test

It parses both lists out of the shipped runner using the existing extractor, and:

• refuses a list it cannot read rather than reporting an empty one — the body group requires at least one character, so local transport_markers='' raises instead of yielding ();
• refuses on zero or two matches (no best-effort pick), on a body containing regex grouping, and on any blank alternative (a bare alternation branch matches every log ever written, which would silently make the whole classifier read artifact);
• asserts the counts are non-zero and anchored to known-shipped strings BEFORE any coverage claim;
• then asserts the covered set equals the shipped set in both directions, naming unproved and orphaned on failure;
• and a further test asserts the mutation sweep itself cannot run zero cases — a parametrised test with no cases reports green.

Three transport alternatives cannot be proved alone because real pip never prints them without a partner (ReadTimeoutError⇄Read timed out, NewConnectionError⇄Failed to establish a new connection). They are declared subsumed and proved as pairs — drop one, still network; drop both, artifact — and the declaration is self-guarding: over-declaring any alternative fails its own drop-one assertion.

FAILURE — three control rods, run by hand against real bash

Rod A — a marker added to the runner with no fixture:

E  AssertionError: transport_markers: shipped but never proved by a fixture: ['BrandNewTransportMarker']
FAILED ...::test_every_shipped_alternative_has_a_fixture

Rod B — a ONE-LETTER typo in a transport marker (IncompleteRead → IncompleteReed). A truncated transfer now blames the wheel, and the suite says so:

E  AssertionError: IncompleteRead
E  assert 'artifact' == 'network'

Rod C — empty the transport list. It must RAISE, never pass vacuously:

E  MarkerListParseError: expected exactly one `local transport_markers='...'` line, found 0

The runner was restored after each rod (git diff clean).

81 tests pass, including the two protected halves of the existing module, which were left byte-identical.

The gate at harvest

PASS  ruff-check          PASS  cf-exemptions        PASS  mypy
PASS  ruff-format         PASS  cf-no-bon-ref        PASS  complexipy
PASS  cf-sticky-check     PASS  cf-import-contract   PASS  pytest
PASS  cf-file-budget      PASS  cf-mirror-check      PASS  cf-recursion-check

cf-gate: PASS — every gate is clean   (exit 0)

LawGiver pre-gate: files_checked=3 added_lines_checked=751, gauges unrun=(none), exit 0.

Two new files because one measured 650 against a 500-line cap. Split by responsibility: the captured corpus (data, no assertions) and the contract over it. tests/scripts has no package total in file-budget.json, so neither adds a draw, and no number was edited. e2e-runner.sh and e2e-box.sh are untouched.

A doc gap found and closed

Four reason codes the runner emits were taught by no playbook: artifact_manifest_missing and artifact_wheel_not_mounted (in release-gates.md but not in the playbook an operator actually opens) and trap:sigint / trap:sigterm (in neither). docs/box-operator.md now carries three one-sentence rows — the trap row states plainly that a killed run says nothing about the artifact — and a test binds all 19 emitted codes to the playbook, with a non-vacuity floor.

🔴 What this does NOT do, and why

It does not get the box to a real PASS, and it does not try. Measured on this machine: docker present (29.6.1), the private fixture checked out — but no .env, no ANTHROPIC_API_KEY, and .e2e-runs/ has never existed. A real run drives bonfire run plus a claude-cli observer session against a live key: it spends the operator's money and copies their OAuth credential into a run directory. That is not mine to authorise. Every test here is offline and deterministic; nothing added can trigger a box run.

So everything downstream of classification remains unproven by this change: that install_step's exit 11 reaches the driver's verdict readout, that emit_failure_verdict writes box_network_unreachable:<step> into a real verdict.json, and that the new playbook rows match what a real killed or unmounted run produces. These tests stub the verdict emitter, exactly as the existing module does.

Findings for a maintainer

1. Structural redundancy in the shipped transport list (a note, not a defect — the runner is untouched). ReadTimeoutError and NewConnectionError cannot be printed by real pip without their partners, so two of seventeen alternatives are unreachable as a sole cause. If the list is ever shortened, those are the only safe removals, and test_a_subsumed_transport_alternative_is_really_subsumed is the proof.
2. No classifier bug found. The ordering, the default-to-artifact fall-through, and the documented honest limit (unreachable index + unresolvable dependency ⇒ network) all behave exactly as the comment claims.
3. file-budget.json self-reports that its gate walks .py files only, so the two .sh entries for e2e-runner.sh (declared debt at 968 against a baseline of 779) and e2e-box.sh are documented but unenforced. A shrink-only ratchet nobody measures is the same class of defect as a marker nobody proved.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com