Skip to content

v0.9.0

Latest
Latest

Choose a tag to compare

View all tags
@cezman cezman released this 20 Sep 23:32
· 110 commits to main since this release
v0.9.0
ddf05c4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ironharness v0.9.0

The external-audit release. Since v0.7.0 the whole pre-0.9 review block (IH-71..82), the full external-audit queue A-D, live-board safety gates, wokwi anti-cheat anchoring, campaign honesty mechanics and the IH-21 benchmark verdict are in.

Live-board safety (audit B)

  • Universal live-board gate: every real-target run/solve requires an explicit --allow-real opt-in - enforced in the dispatcher all callers share, so a named run or a solve can no longer wipe main.py unconfirmed. solve refuses (rc 2) before spending a single LLM token.
  • No wipe without a verified backup: the staging probe must read (or prove absent) the board's main.py - an unreadable board refuses the wipe outright (refusing to wipe main.py without a verified backup).
  • The backup probe waits on a sliding idle deadline (2 s quiet window, 10 s cap): slow boards are no longer misread.
  • main.py restore after a run is documented: flash main.py.backup from the run artifacts back.

Wokwi anti-cheat anchoring (audit A)

  • Position-based boot-dump verdict for the paste scenario: an expect literal printed before the first stimulus echo is condemned as pre-printed; startup output is declared per task via the new boot_expect field (loader-validated).
  • coop-scheduler toggles pins and is scored by the events section - the description-numbers cheater fails the timing checks.
  • The leaderboard carries a standing wokwi/unix non-equivalence note; README documents the boundary (EN + RU).

Env hygiene (audit C)

  • No private-infra default: IRONBENCH_RENODE_DISTRO unset is a loud error naming the variable; paired env tests pin both branches.

Campaign honesty + surface tests (audit D)

  • Tombstone in results.jsonl written atomically before the first attempt: a campaign aborted mid-run can never leave the previous campaign's rows looking fresh; the report skips tombstones.
  • session_status and by-serial open get their first executed tests (empty/full views, resolve + hint).
  • Verbatim cheaters assert the canonical verdict and error_kind=run; a real-target boot-dump offline twin pins the same on the real runner.
  • MpReplError and RuntimeError reach agents as ToolError texts (board tracebacks and refusal hints, not bare crashes).

Bench content

  • New golden tasks: bus-diagnose (bus degradation diagnosis) and debug-station (self-fix loop on the live bench).
  • Bench notes system measured (IH-21 A/B on glm-4.7-flash, live board): 3 lines of bench facts - 4/4 solved on the first iteration vs 2/4 exhausting the budget; tasks uart-echo/uart-menu carry them.
  • README positioning + EN/RU parity; public agent skill; gh-pages leaderboard live.

Packaging

  • sdist hygiene: tool-state and local reports are excluded from the repo and the published artifacts (v0.4.1 precedent closed at the root).
  • From this release on, the WSL distro for staging must be set explicitly: export IRONBENCH_RENODE_DISTRO=<your distro> (no machine-specific default).

Gates: ruff clean, 675+ passed / 55 skipped locally (688/32 with the WSL goldens), every PR subagent-reviewed with empirical red-checks.

Assets 2
Loading