Phase EJ Step 6: Budget-aware final-generation fit + SKILL.md flags documentation

[aberson]

Build step 6 of 8 — Phase EJ (umbrella #281)

Build doc: documentation/plans/evolve-judging-plan.md §7 Step EJ.6

What to build

A generation that starts with insufficient remaining budget dispatches the full pool and dies mid-fitness (_budget_exceeded break at scripts/evolve.py L3542-3549), stranding up to ~25 games of un-actionable results a night.

Behind --budget-fit: before the fitness phase, estimate per-eval wall-clock from this run's completed evals (no observed data on generation 1 → no trim), reserve budget for stack-apply (0 games) + one regression (games_per_eval games) + the gauntlet when --fitness-mode ∈ {baseline, both}, and trim active_idxs to the top-rank prefix that fits — never below 1. Log one phase: "budget_fit" row with the dropped count (no silent caps).

Also: document all six Phase EJ flags in .claude/skills/improve-bot-evolve/SKILL.md's Flags table (EJ.4's pairing warning included).

Existing context

• Budget check _budget_exceeded (scripts/evolve.py L1274-1283); mid-fitness break L3542-3549 strands the generation when the budget dies mid-pool.
• Budget fit trims before dispatch, never below 1. Per-eval cost is estimated from THIS run's own completed evals (self-calibrating — resolves the investigation's "3 min/game vs ~55 games/hour" throughput inconsistency by measuring, not assuming). Fit engages from generation 2.
• This step is the docs roll-up AND serializes the shared-file edits (must run after EJ.1–EJ.5 land in scripts/evolve.py).

Files to modify/create

• scripts/evolve.py (--budget-fit flag + trim logic; budget_fit rows)
• .claude/skills/improve-bot-evolve/SKILL.md (Flags table: all six Phase EJ flags with defaults)
• tests

Done when

Unit tests cover: trim to fitting prefix (rank order preserved); never trims below 1; generation-1 no-op; reserve accounts for regression + gauntlet; flag OFF → byte-identical dispatch. SKILL.md lists all six flags with defaults. uv run pytest / mypy src bots --strict / ruff check . clean; test count ≥ baseline.

Flags (recommended)

--reviewers code

Depends on

#282, #283, #284, #285, #286 (docs roll-up documents all six flags; also serializes the shared scripts/evolve.py edits — must land last)

Parallel-safe with

none — strictly sequential (edits scripts/evolve.py; hard Depends on all prior EJ code steps; plan §4 mandates serial execution)

Produces

--budget-fit flag + trim logic in scripts/evolve.py; budget_fit rows; SKILL.md Flags table update; tests

Operator workflow notes (autonomous + UI-bundle + parallel)

• Runs end-to-end via /build-step; the Done when is the gate.
• No --ui — backend + SKILL.md doc.
• Serial + depends on EJ.1–EJ.5 (last code step).

---

Synced from documentation/plans/evolve-judging-plan.md by /repo-sync at c6099d9

[aberson]

build-phase: Step EJ.6 started (final code step)

Step: Budget-aware final-generation fit + SKILL.md flags documentation
Phase: EJ · Depends on: EJ.1-EJ.5 (all shipped; 5ca62ca)
Flags: --reviewers code (--isolation none)
Baseline tests: 1938 passing
Baseline HEAD: 5ca62ca (EJ.5)
Started: 2026-07-06

Behind --budget-fit: before each fitness phase (gen 2+), estimate mean per-eval wall-clock from THIS run's completed evals (self-calibrating), reserve budget for one regression eval + the gauntlet (when --fitness-mode baseline/both), and trim active_idxs to the top-RANK prefix that fits the remaining --hours — never below 1. Gen 1 = no-op (no observed data). One budget_fit audit row with the dropped count (no silent caps). Plus: SKILL.md Flags table documenting all six Phase-EJ flags with defaults + the EJ.4 pairing warning. Default OFF → byte-identical dispatch.

[aberson]

build-phase: Step EJ.6 complete (final code step)

Verdict: PASS (iteration 2/3)
Commit: 2c8190f
Files changed: scripts/evolve.py, .claude/skills/improve-bot-evolve/SKILL.md, tests/test_evolve_cli.py, tests/test_evolve_parallel.py
Tests: 1952 passing (+14 over 1938 EJ.5 baseline)
Gates: mypy strict OK (808 files), ruff check OK, ruff format --check (no new debt)
Reviewers: 4 code — bugs CLEAN (all degenerate inputs probed), correctness (2 low), test-quality (1 med + 2 low), style (2 low). Iteration 2 closed the medium + naming/doc lows.

Changes made

• --budget-fit (default OFF → byte-identical): before each generation ≥2's fitness phase, estimate mean per-eval wall-clock from THIS run's completed evals (self-calibrating), reserve one regression eval + the gauntlet (when --fitness-mode baseline/both), and trim active_idxs to the top-RANK prefix that fits the remaining --hours — never below 1. Gen 1 = no-op (no observed data). One budget-fit audit row with the dropped count (no silent caps).
• Estimator (eval_durations_seconds) is fed by BOTH the serial and parallel dispatch paths — one accumulator, ONLY real games-played evals (null-diff screens and crashes excluded; poisoning guard pinned by 4 new tests so a future refactor can't silently shrink the mean and reintroduce the strand).
• Conservative under --concurrency>1 (per-worker wall-clock modeled as serial → errs safe, never strands; documented at the trim block).
• SKILL.md: comprehensive six-flag Phase-EJ table (EJ.1–EJ.6) with types/defaults + the one-sided↔panel-floor pairing warning (reconciled to appear once).

Review findings addressed (iteration 1 → 2)

• MED (test): the estimator poisoning guard was untested → test_budget_fit_estimator_excludes_null_diff_and_crash_serial (asserts the mean is the single success's 600s, not a 200s poisoned mean) + 3 parallel tests (..._null_diff_does_not_feed_estimator, ..._crash_does_not_feed_estimator, ..._success_feeds_estimator_once).
• LOW: renamed eval_durations → eval_durations_seconds (unit-suffix convention); reworded help text (trims every gen ≥2, not just the final); added the --concurrency>1 conservatism comment.

Byte-identical off verified: --budget-fit off → full active set dispatched (may strand, today's behavior), no budget_fit row. Strand-prevention proven through the real run_loop (ON → generations-reached, OFF → wall-clock strand).

---

Phase EJ automated span (EJ.1–EJ.6) COMPLETE. All six flags default OFF (byte-identical); the one deliberate default-visible change is EJ.1's anti-verbatim priors reword. Remaining: EJ.7 (operator real-SC2 smoke) + EJ.8 (overnight A/B soak) — operator handoff.

[aberson]

Completed via /build-phase (EJ.6, commit 2c8190f). Gates green: pytest 1952, mypy strict, ruff. Final code step of Phase EJ. Estimator poisoning guard pinned by 4 tests. EJ.1-EJ.6 automated span complete; EJ.7/EJ.8 are operator handoff.