v0.9.0

ironharness v0.9.0

The external-audit release. Since v0.7.0 the whole pre-0.9 review block (IH-71..82), the full external-audit queue A-D, live-board safety gates, wokwi anti-cheat anchoring, campaign honesty mechanics and the IH-21 benchmark verdict are in.

Live-board safety (audit B)

• Universal live-board gate: every real-target run/solve requires an explicit --allow-real opt-in - enforced in the dispatcher all callers share, so a named run or a solve can no longer wipe main.py unconfirmed. solve refuses (rc 2) before spending a single LLM token.
• No wipe without a verified backup: the staging probe must read (or prove absent) the board's main.py - an unreadable board refuses the wipe outright (refusing to wipe main.py without a verified backup).
• The backup probe waits on a sliding idle deadline (2 s quiet window, 10 s cap): slow boards are no longer misread.
• main.py restore after a run is documented: flash main.py.backup from the run artifacts back.

Wokwi anti-cheat anchoring (audit A)

• Position-based boot-dump verdict for the paste scenario: an expect literal printed before the first stimulus echo is condemned as pre-printed; startup output is declared per task via the new boot_expect field (loader-validated).
• coop-scheduler toggles pins and is scored by the events section - the description-numbers cheater fails the timing checks.
• The leaderboard carries a standing wokwi/unix non-equivalence note; README documents the boundary (EN + RU).

Env hygiene (audit C)

• No private-infra default: IRONBENCH_RENODE_DISTRO unset is a loud error naming the variable; paired env tests pin both branches.

Campaign honesty + surface tests (audit D)

• Tombstone in results.jsonl written atomically before the first attempt: a campaign aborted mid-run can never leave the previous campaign's rows looking fresh; the report skips tombstones.
• session_status and by-serial open get their first executed tests (empty/full views, resolve + hint).
• Verbatim cheaters assert the canonical verdict and error_kind=run; a real-target boot-dump offline twin pins the same on the real runner.
• MpReplError and RuntimeError reach agents as ToolError texts (board tracebacks and refusal hints, not bare crashes).

Bench content

• New golden tasks: bus-diagnose (bus degradation diagnosis) and debug-station (self-fix loop on the live bench).
• Bench notes system measured (IH-21 A/B on glm-4.7-flash, live board): 3 lines of bench facts - 4/4 solved on the first iteration vs 2/4 exhausting the budget; tasks uart-echo/uart-menu carry them.
• README positioning + EN/RU parity; public agent skill; gh-pages leaderboard live.

Packaging

• sdist hygiene: tool-state and local reports are excluded from the repo and the published artifacts (v0.4.1 precedent closed at the root).
• From this release on, the WSL distro for staging must be set explicitly: export IRONBENCH_RENODE_DISTRO=<your distro> (no machine-specific default).

Gates: ruff clean, 675+ passed / 55 skipped locally (688/32 with the WSL goldens), every PR subagent-reviewed with empirical red-checks.

v0.7.0

Observability + gates (IH-8, IH-29, IH-30, IH-20)

Journal viewer. ironbench journal <file> renders a self-contained offline HTML timeline of a run journal, with kind/actor/connection filters and text search. Hardened against hostile journal content: template-token injection, self-overwrite, NaN/Infinity, deep nesting.

io-core gates and failure journaling (IH-29). The IRONHARNESS_ALLOW_REAL_FLASH safety gate is checked first (before image-path and optional-dependency checks) and every denial is journaled (esp_denied) - a gate without a negative test is decorative, so the deny path is pinned by tests. Failure journaling covers the whole transport family: modbus open/read/write (including dropped-link ConnectionException), mqtt open/publish/subscribe, serial open/write/read/read_line; file_list is journaled; esp_image_info passes chip through. Windows NTFS-junction sandbox-escape test added.

MCP stdio smoke + LLM HTTP layer (IH-30). The real python -m io_core.mcp_server subprocess is driven over the real wire by two independent clients (a hand-rolled line-delimited JSON-RPC client and the official SDK ClientSession); the packaging CI job pings the installed wheel. agent.chat is tested against a local OpenAI-compatible http.server (request shape, content extraction, typed errors, HTTP-500 -> solve infra classification); the LLM config priority chain (explicit > env > .env > default) is pinned.

MCP tool annotations + stdout hygiene (IH-20). All 22 tools declare explicit ToolAnnotations (readOnly only where nothing is consumed; draining reads are not readOnly; destructive only for esp_flash/esp_erase/file_delete/file_write; idempotentHint deliberately unset). The stdio wire is pinned pure: an error battery keeps stdout JSON-RPC-only, and a deliberately noisy handler must land on stderr.

Runner hygiene (merged on the way to the real-hardware track). runner.py split into per-target modules behind a stable facade (runner_common + wokwi/renode/unix/plant/real); _drive_repl returns a structured error_kind; the unix anti-cheat verdict is now a deterministic post-run classification over the final log (previously the wording raced the reader thread); env pins (IRONBENCH_UNIX_BIN, IRONBENCH_RENODE_BIN, WOKWI_CLI_TOKEN priority) covered by tests.

Honest limits (documented, from the pre-release adversarial pass). One journal writer per file: concurrent threads of a session are safe, but two processes sharing one IRONHARNESS_HOME can silently drop lines - keep one agent per home directory (a cross-process lock is queued for IH-32).

Testing: 424 passed / 26 skipped (CI: ubuntu + windows + packaging smoke of the installed wheel). Pre-release adversarial pass over the whole harness found no blockers.

ironharness 0.6.0 — benchmark-honesty block (IH-14..17)

Second reliability block from the 2026-09-09 audit: the benchmark's verdicts and the convention 'transports always have timeouts and quotas' are now enforced and pinned by tests.

anticheat (IH-14)

• A wait-serial answer that arrives stamped before the stimulus write is pre-printed output - the run fails. A firmware that exits before answering a single stimulus step while matching the patterns fails too.
• Plant tasks carry mid-run ambient disturbances: a tuned open-loop constant power drifts out of band and fails; feedback controllers track through.
• The anticheat suite is parametrized over every task - new tasks are covered automatically.

journal + solve (IH-15)

• TaskResult carries a structured error_kind (none/infra/timeout/run); solve's early exit reads the kind, never the error text - firmware printing 'not found' no longer stops a campaign.
• solve feedback shows the failure reason.

machine shim + honest blink (IH-16)

• The unix target can run GPIO-style firmware through a harness-provided machine shim; every pin transition is logged with a timestamp.
• New 'events:' scoring: transition count and every consecutive period must fit the declared bounds - printing the expected strings does not pass. Golden blink-unix included.
• 'ironbench clean' removes old run artifacts, only from ironbench-marked directories.

io-core session (IH-12/IH-17)

• Atomic transport registry and sandbox quotas (parallel opens/writes cannot slip past checks), robust close, IRONHARNESS_MAX_CONNECTIONS.
• Deadline and rate limits applied to session transports by default: IRONHARNESS_TRANSPORT_DEADLINE (600 s default) and IRONHARNESS_TRANSPORT_RATE=max/window.
• ALLOWED_HOSTS rejects IPv6 literals loudly instead of silently never matching.

Full suite: 327 passed / 23 skipped; two protocol subagent review rounds per PR.

ironharness 0.5.0 — reliability block (IH-10..13)

Reliability block from the 2026-09-09 audit — the benchmark's PASS, its journal, and its concurrency guarantees are now backed by pinned tests.

ironbench

• PASS fidelity (IH-10): every run writes artifacts into a fresh out_dir/<task>/run-<id>/ directory; the plant worker stamps result.json with a run id and the runner rejects foreign/stale reports. A re-run into a dirty out dir can no longer score a previous run's output.
• Throwaway controller cwd (IH-13): plant controllers run with a scratch working directory; relative writes stay inside the run artifacts.
• ironbench clean (new): removes old run-* dirs (keeping the newest per task) and refuses directories without the .ironbench-out marker — it only ever cleans what ironbench created.
• Honest threat model documented: task code (plant controllers, unix/real firmware) runs as a local process with your rights — a process is not a sandbox.

io-core

• Journal attribution (IH-11): every transport event now carries the connection name (conn); ReplaySession replays each connection as its own stream with per-connection strict write verification (legacy journals without conn replay as before).
• Concurrency (IH-12): the transport registry and sandbox quotas are atomic under a lock (duplicate opens and parallel quota-busting writes are gone), get_session() is race-free, Session.close() survives a stuck port, and a new IRONHARNESS_MAX_CONNECTIONS policy caps open transports (denials are journaled).

Full suite: 293 passed / 22 skipped; every fix is pinned by a test that fails without it.

v0.4.3

English churn of ironbench (IH-9): runner/plant/report/CLI strings, task descriptions and old comments are now English; behavior unchanged. Also: a missing IRONBENCH_REAL_PORT now classifies as an infra error (early exit instead of burning agent iterations).

v0.4.2

Housekeeping release - same code as v0.4.1 (access policy: IRONHARNESS_ALLOWED_HOSTS, IRONHARNESS_ENABLED_KINDS, MCP close tools), with clean packaging. Prefer 0.4.2 over 0.4.1.

v0.4.0 - real-hardware target

• New ironbench target: real - the same golden tasks run on a live board (MicroPython REPL over USB-UART), opt-in via IRONBENCH_REAL_PORT.

IRONBENCH_REAL_PORT=COM4 uv run ironbench run --task uart-echo --target real

• CH340-safe serial IO: rate-limited writes, single-threaded link access (bursts corrupt into NULs, concurrent read+write crashes the driver)
• paste-mode handshake, main.py hygiene + soft reset before staging, output buffer cleared after staging
• live-verified on ESP32: uart-echo PASS in 20.9s

v0.3.2 — MCP close tools

• New MCP tools: serial_close, modbus_close, mqtt_close — release ports without restarting the server (22 tools total).
• Typed close: closing a name bound to another transport kind raises a clear error.
• First release shipped via OIDC trusted publishing (no API tokens).