v0.2.1 — codex adapter live-validated
Live validation on a real harness 🎯
The codex adapter is now live-validated (codex-cli 0.149.0-alpha, gpt-5.6-sol): the first preregistered controlled A/B on a real agent.
What the live run produced (4 tasks × 2 variants, description-clarity A/B):
| arm | subject selection | ops | success | consumption | real tokens/op | median steps |
|---|---|---|---|---|---|---|
| control (plain description) | 0/4 | 4 | 100% | 4/4 | 188,520 | 9.5 |
| clear (rewritten description) | 1/4 | 4 | 100% | 4/4 | 215,021 | 22.5 |
Verdict: honest null (+25pp observed at n=4/arm, p=0.29) with next-round sizing guidance (≈31/arm) — exactly the discipline the engine exists to enforce. A weak default description losing to better-described competitors matches Hasan et al. (arXiv 2602.14878).
Fixes found by the live run (scripted tests could not catch these): mcp_tool_call item parsing; --approve-for-me for MCP calls in exec mode; stdin DEVNULL; toolserver contract fix (spec id→name, crashed tools/list) with a contract regression test; candidate-set steering in the episode prompt; token usage metered as cost units (1 unit = 1 token) so the budget breaker bounds real spend.
claude-code shares these fixes and remains scripted-transcript-tested pending its own live run. 78 tests.