Phase EJ Step 4: Frozen-baseline panel floor

[aberson]

Build step 4 of 8 — Phase EJ (umbrella #281)

Build doc: documentation/plans/evolve-judging-plan.md §7 Step EJ.4

What to build

EJ.3's honest cost is regression catch at p=0.35 dropping 76.5% → 42.8%, and the "caught later" backstop is false (the next generation compares against the same weakened bar) — so the one-sided rule needs a drift backstop that cannot cycle. The EL gauntlet already plays the promoted version vs each frozen anchor under --fitness-mode both and is explicitly log-only ("a later step can drive gating off it", scripts/evolve.py L3825-3832). This is that later step — consume the existing gauntlet, add no games.

Behind --panel-floor: when the gauntlet ran and any GauntletResult.per_baseline win rate is 0.0 (sweep loss vs any anchor), trigger the existing rollback machinery (revert-first ordering per L4030-4063) with reason prefix panel-floor: and flip promoted imps to regression-rollback.

Fail-open by construction: gauntlet crash, empty registry, or --fitness-mode parent → floor inert (the existing defense-in-depth try/except at L3837-3923 must keep protecting the committed promotion — a monitoring layer must never block an already-committed promotion on its own failure). Add a startup log line when panel-floor is on but the registry is empty.

Document in SKILL.md that production soaks should not enable --regression-rule one-sided without --panel-floor + registered baselines.

Existing context

• Gauntlet log-only block: scripts/evolve.py L3825-3832 (comment: "The gauntlet does NOT change the promotion gate in this step — it LOGS + records ... so a later step can drive gating off it"), wrapped defense-in-depth L3837-3923.
• Rollback consumption: scripts/evolve.py L4030-4063 — if regression_result.rolled_back: flips promoted imps to _REGRESSION_ROLLBACK, revert-first ordering (git revert on a clean tree restores the pointer via reverse diff), pointer-restore fallback.
• run_baseline_gauntlet (src/orchestrator/evolve.py L847-976) returns GauntletResult.per_baseline win rates + mean_win_rate; skips a baseline whose version IS the candidate; empty registry → mean_win_rate=0.0.
• Joint null false-positive with the posterior bar ≈ 26% at K=3 anchors (each anchor sweep-loss ≈ 3.1% under fair coin) — but frozen anchors cannot cycle, so sustained drift cannot pass them (AlphaStar max-min rationale).
• data/baselines.json is ABSENT today — EJ.7 registers anchors; unit tests use fakes. Do NOT create it early.

Files to modify/create

• scripts/evolve.py (--panel-floor flag + floor consumption; panel-floor: rollback rows; startup log when on-but-empty)
• .claude/skills/improve-bot-evolve/SKILL.md (pairing warning: one-sided ⇒ requires panel-floor + baselines)
• tests

Done when

Unit tests with fake gauntlet results cover: sweep loss vs one anchor → rollback via the same revert path; all anchors ≥ 1 win → no effect; gauntlet crash → promotion stands (fail-open); flag OFF → byte-identical. Integration test drives the loop with fitness-mode both + fake baselines and asserts floor rollback lands a panel-floor: reason row. uv run pytest / mypy src bots --strict / ruff check . clean; test count ≥ baseline.

Flags (recommended)

--reviewers code

Depends on

#284 (EJ.3 — reuses the one-sided rollback machinery / is the mandatory pairing for the relaxed regression bar)

Parallel-safe with

none — strictly sequential (edits scripts/evolve.py shared with sibling EJ steps; also has a hard Depends on: EJ.3; plan §4 mandates serial execution)

Produces

--panel-floor flag + floor consumption in scripts/evolve.py; panel-floor: rollback rows; SKILL.md pairing warning; tests

Operator workflow notes (autonomous + UI-bundle + parallel)

• Runs end-to-end via /build-step; the Done when is the gate.
• No --ui — backend + SKILL.md doc.
• Serial + depends on EJ.3.

---

Synced from documentation/plans/evolve-judging-plan.md by /repo-sync at c6099d9

[aberson]

build-phase: Step EJ.4 started

Step: Frozen-baseline panel floor
Phase: EJ · Depends on: EJ.3 (shipped becf306)
Flags: --reviewers code (--isolation none)
Baseline tests: 1909 passing
Baseline HEAD: becf306 (EJ.3)
Started: 2026-07-06

Behind --panel-floor: when the EL baseline gauntlet ran and any per_baseline win rate is 0.0 (sweep loss vs any frozen anchor), combine that into the regression-phase rollback decision (rolled_back OR panel_floor_swept) — reusing the existing git-revert-first machinery, reason prefix panel-floor:. Fail-open on gauntlet crash / empty registry / --fitness-mode parent. Mandatory pairing for EJ.3's one-sided rule (SKILL.md warning). Default OFF → byte-identical.

[aberson]

build-phase: Step EJ.4 complete

Verdict: PASS (iteration 2/3)
Commit: d69ae6a
Files changed: scripts/evolve.py, .claude/skills/improve-bot-evolve/SKILL.md, tests/test_evolve_cli.py
Tests: 1923 passing (+14 over 1909 EJ.3 baseline)
Gates: mypy strict OK (808 files), ruff check OK, ruff format --check (no new debt)
Reviewers: 4 code — correctness CLEAN (7 invariants line-verified), style (2 low), bugs (1 med), test-quality (1 med + 3 low). Iteration 2 closed both mediums + all lows.

Changes made

• --panel-floor (default OFF → byte-identical): when the EL baseline gauntlet shows a sweep loss (any per_baseline win rate == 0.0 vs a frozen anchor), the promotion is rolled back through the existing regression revert machinery (refactored into a shared _apply_rollback helper — single revert path, no double-revert), reason prefix panel-floor:. Combined into the regression-phase decision (regression_result.rolled_back OR panel_floor_swept); regression reason takes precedence when both fire.
• Backstop holds on regression crash: the sweep signal is computed in the gauntlet BEFORE and independently of the regression eval, so a regression-eval crash (SC2 hang / watchdog kill) no longer strands a swept version as the live parent — the floor enforces even on the crash path (while staying fail-open for the gauntlet's OWN failures).
• Fail-open: gauntlet crash / empty registry / --fitness-mode parent → floor inert. Armed-but-inert startup WARNING when --panel-floor is on but the config makes it inert (operator-safety).
• Per-generation reset of panel_floor_swept (no state leak across generations).
• SKILL.md pairing warning: production soaks should not run --regression-rule one-sided without --panel-floor + registered frozen baselines.

Review findings addressed (iteration 1 → 2)

• MED (bug): panel floor was bypassed when the regression eval crashed → now enforced in the crash branch via the shared _apply_rollback; test test_panel_floor_enforces_on_regression_crash.
• MED (test): no multi-generation state-leak guard → test_panel_floor_state_resets_across_generations (gen1 sweeps→rolls back, gen2 clean→stands, exactly one panel-floor row).
• LOW: added test_panel_floor_inert_when_both_mode_but_zero_baselines; tightened the precedence test (generations_promoted==0); dropped redundant default=False; converted the SKILL.md blockquote to the doc's bold-paragraph convention.

Byte-identical off verified across the full (rolled_back × sweep) grid. This is the mandatory pairing for EJ.3's one-sided rule.

[aberson]

Completed via /build-phase (EJ.4, commit d69ae6a). Gates green: pytest 1923, mypy strict, ruff. Panel floor now enforces even on a regression-eval crash (backstop cannot cycle). Mandatory pairing for EJ.3 one-sided. See result comment above.