Follow-up from #649 (per your merge note: "file it as its own issue and it'll get picked up — including by you").
What
A committed self-test that proves the #648 runaway guard END-TO-END on every suite run — not just when someone hand-drives a scratch file through check_eigs_suite:
- A committed 3-line runaway fixture, e.g.
tests/_runaway_guard_selftest.eigs:
i is 0
loop while 1 == 1:
i is i + 1
- A dedicated runner block that invokes
check_eigs_suite on it with a tiny per-call override (EIGS_TEST_TIMEOUT=2) so the whole demonstration costs ~2s, not 180.
- Inverted expectation: the block treats guard-fired (the named
timed out after 2s — runaway in … failure, rc=124 path) as its PASS, and anything else — the fixture terminating, a mis-named failure line, a wrong tally increment — as FAIL. The suite's total tally counts the self-test as one ordinary passing check.
Why the inversion is the whole trick
Without it, the guard firing correctly would count as a suite failure — which is why #649 left this out as "more runner surgery than #648 asked for." The inversion needs a small seam: either a variant of check_eigs_suite that expects rc=124 (asserting the failure line's shape and that FAIL was incremented by exactly the declared count, then converting the result), or a self-contained block that snapshots PASS/FAIL before/after and asserts the deltas. The tally-exactness assertion matters as much as the firing — your own #649 verification nearly flagged a phantom FAIL=2 from an over-extracting harness; a committed self-test makes that class of doubt permanent instead of per-review.
Cost
~2s per suite run (the override bounds it); no effect on the 180s production budget; degrades to a skip-with-reason if timeout/gtimeout are both absent (the same degradation the guard itself has).
Happy to send the PR — it slots on top of the merged #649 wiring.
Filed with AI assistance (Claude); the design was sketched and scoped during #649.
Follow-up from #649 (per your merge note: "file it as its own issue and it'll get picked up — including by you").
What
A committed self-test that proves the #648 runaway guard END-TO-END on every suite run — not just when someone hand-drives a scratch file through
check_eigs_suite:tests/_runaway_guard_selftest.eigs:check_eigs_suiteon it with a tiny per-call override (EIGS_TEST_TIMEOUT=2) so the whole demonstration costs ~2s, not 180.timed out after 2s — runaway in …failure, rc=124 path) as its PASS, and anything else — the fixture terminating, a mis-named failure line, a wrong tally increment — as FAIL. The suite's total tally counts the self-test as one ordinary passing check.Why the inversion is the whole trick
Without it, the guard firing correctly would count as a suite failure — which is why #649 left this out as "more runner surgery than #648 asked for." The inversion needs a small seam: either a variant of
check_eigs_suitethat expects rc=124 (asserting the failure line's shape and that FAIL was incremented by exactly the declared count, then converting the result), or a self-contained block that snapshots PASS/FAIL before/after and asserts the deltas. The tally-exactness assertion matters as much as the firing — your own #649 verification nearly flagged a phantomFAIL=2from an over-extracting harness; a committed self-test makes that class of doubt permanent instead of per-review.Cost
~2s per suite run (the override bounds it); no effect on the 180s production budget; degrades to a skip-with-reason if
timeout/gtimeoutare both absent (the same degradation the guard itself has).Happy to send the PR — it slots on top of the merged #649 wiring.
Filed with AI assistance (Claude); the design was sketched and scoped during #649.