Skip to content

Harden the timeout-budget guard: write-once slots, exact counts, comment cross-checks - #35

Merged
oratis merged 2 commits into
mainfrom
opt/ci-timeout-budget-v2
Aug 2, 2026
Merged

Harden the timeout-budget guard: write-once slots, exact counts, comment cross-checks#35
oratis merged 2 commits into
mainfrom
opt/ci-timeout-budget-v2

Conversation

@oratis

@oratis oratis commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Follow-up applying the deep-review fixes for PR #31, which merged before they could land.

What the review found (mutation-verified) and what this fixes

  1. Silent budget reassignment — a job-id line with a trailing comment (publish: # optional) was invisible to the scanner, so its timeout-minutes overwrote the previous job's slot: the guard then checked lifecycle against 300 and passed. Job-id/timeout/TimeoutStartSec/profile_timeout_seconds patterns now tolerate trailing comments, and timeout slots are write-once (a second write aborts naming the unrecognized line).
  2. Counts were unbounded downward — commenting out a run_profile (3→2) passed green. The guard now parses scenarios=N and N guest boots out of the harness scripts' own output lines and asserts mutual agreement with the counted call sites/markers.
  3. The 13 ASSUMED constants were a second copy of the workflow comments with nothing comparing them — the exact "second source of truth" failure the script's header promises to avoid. New parse_budget_comments extracts each job's = N min < N min (here) summary and asserts it equals the derived total and parsed timeout (fail-closed on missing/duplicate lines). Editing a comment total by 1 now fails CI.
  4. One-sided assertiontimeout-minutes: 360 passed. Now: headroom ≤ 15 min and job timeout ≤ 240.
  5. TimeoutStartSec=0 (systemd: no timeout) now aborts; octal-leading-zero arithmetic fixed with 10#; vacuous invariant honestly reworded; wrong indentation-claim comment corrected; grep tripwire added against the six historical 180m drift references (all resolved by Cache payload layers across runs and split build from lifecycle #28's fix; the tripwire keeps them gone).
  6. ci.yml: guard moved before the cargo steps (it only needs checkout — feedback in seconds, and it still runs when a Rust test fails); shellcheck added to the quality job over the full os-e2e validate file set (every file verified passing).

Mutation transcript in the guard's own terms: unmodelled third job → hard fail; unrecognizable job id → write-once backstop fail; removed profile → count mismatch fail; comment total off by one → cross-check fail. On main: ANDROMEDA_CI_TIMEOUT_BUDGET_OK build=71/75 lifecycle=135/140 guest=42/45.

🤖 Generated with Claude Code

…-checks, bounded headroom

Addresses the deep-review comment on PR #31.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oratis

oratis commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

The lifecycle failure here is not caused by this PR — I diagnosed it while sweeping open PRs.

The guest completed the whole lifecycle. ANDROMEDA_E2E_OK is present in the serial evidence, and so are the other nine markers. The run failed because one of them arrived cut in half:

ANDROMEDA_SELINU[    8.687420] snd_hda_codec_generic hdaudioC0D0:      Line=0x5
X_LABELS_OK

The guest and the kernel share one UART, and a printk landed mid-write. The validator requires each marker exactly once, counted zero for ANDROMEDA_SELINUX_LABELS_OK, and failed. Existing normalization strips NUL/CR/ANSI and cannot repair it, because the marker is split by another writer's text rather than by control characters.

This is timing-dependent and can hit any PR — nothing on this branch touches that path.

#37 fixes it by rejoining mid-line kernel splices, verified against this run's own boot-serial.log (that marker goes 0 matches → 1, and all ten then pass the exactly-once and ordering checks). Re-running this PR once #37 lands should clear it.

@oratis
oratis merged commit 3f8694c into main Aug 2, 2026
7 checks passed
@oratis
oratis deleted the opt/ci-timeout-budget-v2 branch August 2, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant