v0.17.2

0.17.2 (2026-10-30)

Bug Fixes

• cli: read a passed-on Ctrl-C as one press whichever signal arrives first (#738) (3c02c33)
• harness: record unmeasured reasoning time as null and count subagent edits (#736) (0ee5467)

v0.17.1

0.17.1 (2026-10-30)

Bug Fixes

• antigravity: map manage_task, agy 1.4.7's background-task tool (#722) (bf646ac)
• cli: keep the run record when a launcher passes Ctrl-C on, and say what a second press skips (#730) (4898cee)
• cursor: record tool calls with no result as unfinished, not failed (#731) (b8d3c6e)
• egress: record and report a refused brokered request (#717) (ea08bb7)
• egress: record names that don't resolve apart from policy blocks (#729) (0b330fe)
• goose: keep the image's bootstrap at every effort, so Goose isn't reinstalled at each attempt (#721) (a010453)
• goose: pass Vertex AI's variables through a launcher, so the eval's own reach the agent (#725) (2738106)
• harness: broker Antigravity's web search endpoint so its searches land (#718) (9e6a7af)
• harness: close the open step before finish when a turn is cut short (#726) (0ca2a70)
• harness: make a key the model API rejects in a turn an infra error (#728) (fda3b46)
• harness: send OpenCode no effort on a model whose API refuses one (#724) (f17d42f)
• install: install the binary that never ran, not the one the start check ran (#733) (a1da280)
• opencode: keep the image's bootstrap on every provider, so OpenCode isn't reinstalled at each attempt (#723) (366969f)
• stop a canceled capture waiting for chrome, and keep digests whole for short secrets (#719) (47dbcec)

v0.17.0

0.17.0 (2026-10-30)

Features

• capture: end an instruction on the move that completes it, and keep each instruction's screenshot (#712) (a8ea7b2)
• capture: expose page facts on c.capture() and add toHaveStatus and toHaveNoPageErrors (#704) (d8349e1)
• ui: show the run header as one block on a color terminal (#713) (06141de)

Bug Fixes

• capture: read a packed point's missing half from its own field (#711) (d7fa2d4)
• harness: broker Codex's web search endpoint so its searches land (#715) (355b5f5)
• harness: run every harness at medium effort when unset and check Cursor's catalog at plan time (#714) (ee1d48a)

v0.16.1

0.16.1 (2026-10-29)

Tests

• hold the matcher-line and readTree symlink tests to Windows paths (#708) (6b06352)

v0.16.0

0.16.0 (2026-10-29)

⚠ BREAKING CHANGES

• history: snapshot each attempt's workspace into its run directory (#700)
• expect: make expect soft by default and add expect.require (#701)
• judge: make every judge input explicit (#702)

Features

• capture: route the page's loopback requests into the sandbox (#706) (7d4180e)
• expect: make expect soft by default and add expect.require (#701) (434f1b5)
• history: snapshot each attempt's workspace into its run directory (#700) (a5dfc23)
• judge: make every judge input explicit (#702) (e2e63a5)
• runs: add runs: { bestOf: n }, stopping each cell once a majority agrees (#690) (5518351)
• serve: find the port c.serve's command listens on and wait for it (#703) (697b92e)

Bug Fixes

• capture: make interactive c.capture finish ordinary multi-step flows (#705) (fe8b36a)
• expect: type each matcher to its subject and check an awaited exec at once (#699) (c7e2238)
• fx: bind the relay test to port 0 and wait for the port it listens on (#693) (a8ac5d3)

v0.15.1

0.15.1 (2026-10-27)

Bug Fixes

• pi: link the fake bootstrap with a junction so the gateway tests run on Windows (#691) (055574b)

v0.15.0

0.15.0 (2026-10-27)

⚠ BREAKING CHANGES

• plan: run a compare 4 times a side when runs is unset (#687)
• gateway: end every gateway list with direct(), with direct({ enabled: false }) for strict routing (#676)
• expect: fail the attempt on a caught failed expect (#684)
• spring clean: remove insights, single-source option docs and trim slow or duplicate tests (#667)
• history: fingerprint an eval's body, options, cases and revision (#664)
• updated en grammar to us grammar (e.g. JudgmentRecord, judgment_key, canceled)

Features

• doctor: check vertex() models online, with a Vertex example and recipe (#674) (bc75f37)
• gateway: accept the Anthropic API's dated Claude ids on Vertex AI (#672) (47958ee)
• gateway: add openrouter() routing gateway (#547) (0778bbc)
• gateway: end every gateway list with direct(), with direct({ enabled: false }) for strict routing (#676) (be3c673)
• gateway: run Antigravity, OpenCode, Goose and pi on Vertex AI (#673) (4d0e7c9)
• gateway: run fx on openrouter() (#680) (9c3b3b6)
• gateway: run fx on Vertex AI (#686) (238abff)
• gateway: run Grok Build on Vertex AI (#675) (31a8cd2)
• gateway: run pi on openrouter() (#679) (1830a9b)
• gateway: run pi on vercel() and send effort through gateways where the model reads it (#671) (52280a5)
• gateway: vertex(), with Claude Code, judges and analysis on Vertex AI (#505) (b0c3690)
• history: fingerprint an eval's body, options, cases and revision (#664) (fdb040b)
• plan: hint the compares and families that can't get a verdict (#685) (b478ba9)
• plan: run a compare 4 times a side when runs is unset (#687) (74df7c8)
• query: name a mixed cell below the flaky minimum mixed (#681) (d3a1e78)

Bug Fixes

• accept files saved with a UTF-8 BOM wherever reeval reads user JSON, frontmatter or .env (aefe0c6)
• attempt: keep assertion labels unique when a given label looks numbered (aefe0c6)
• attempt: record a judge error only when a judge caused the failure (aefe0c6)
• capture: accept only capture names the studio can serve (aefe0c6)
• cli: check the keys of group and eval judges before the run (aefe0c6)
• cli: switch out of cancel mode as live.json turns finishing (#548) (e84168a)
• diff: keep changed paths as written and count hunk lines that start with -- or ++ (aefe0c6)
• engine: exit a worker at once when the CLI goes while its file is loading (aefe0c6)
• engine: fail a file's attempts when it fails to load in the run pass (af665de)
• engine: let a filter part that names a file also match names and cells (6f763ad)
• engine: run up to concurrency eval files at once, not one per cpu (6f763ad)
• engine: skip .only evals inside a skipped group in the plan (aefe0c6)
• environment: refuse a password or token in skill and npm source URLs (af665de)
• evaluate: refuse evals and pooled families that would share an id (aefe0c6)
• expect: fail the attempt on a caught failed expect (#684) (68c3447)
• harness: record writes of unknown origin as write, not modify (6f763ad)
• harness: refuse MCP names and values opencode would substitute (6f763ad)
• history: claim a run before writeRun touches its artifact directory (#559) (4514416)
• history: compare pooled-family gates and fail on their regressions (#562) (d229233)
• history: count attempts with no tool calls in effort's per-class medians (aefe0c6)
• history: count gateway changes as drift, not noise (#560) (bd7a243)
• history: keep each failed attempt's artifacts with artifacts 'failures' (#561) (6c4ce39)
• history: keep minLift, provenance and pairwise answers on resume (#563) (00b74e8)
• history: key environments and env_skills on name and digest (aefe0c6)
• history: name a gate change in compare instead of calling it a regression (#565) (6239b12)
• history: settle run commits past rev-list's 20,000 with merge-base (aefe0c6)
• judge: grade a language runner's judge on value as the value itself (aefe0c6)
• judge: mask the judge's input before it's sent (aefe0c6)
• judge: number judgement keys by call order, not completion order (aefe0c6)
• pi: carry a turn on through the run pi continues after compacting (aefe0c6)
• pi: keep a turn pi recovered by auto-retry (aefe0c6)
• profiles: refuse a limit.harness id no eval runs on ([6f763ad](https://github.com/invertase/ree...

v0.14.0

0.14.0 (2026-10-24)

⚠ BREAKING CHANGES

• move questions to the eval's root and ask them of each attempt (#543)
• gateway: vercel() for Claude Code and Codex, and gateway-routed judges (#503)
• egress: default to 'proxy', which sends everything through the egress gateway (#501)

Features

• checks: retry c.exec on a non-zero exit with retries and interval (#542) (311cd92)
• cli: estimate cost as a range in reeval run --dry-run (#519) (4e18100)
• egress: default to 'proxy', which sends everything through the egress gateway (#501) (0cc02cb)
• gateway: add the gateway list with direct(), plan-time resolution and pins (#502) (22e4d5a)
• gateway: run Goose, OpenCode, fx and Grok Build on Vercel AI Gateway (#504) (4d89bed)
• gateway: vercel() for Claude Code and Codex, and gateway-routed judges (#503) (9128d66)
• language-runner: add a record call for checks made in the eval's own language (#511) (c0dfacf)
• move questions to the eval's root and ask them of each attempt (#543) (c5f0ab7)

Bug Fixes

• engine: keep a worker up until the main process has its last message (#524) (cfa2fd5)
• install: detect musl from the libc in use (#522) (d9990e1)
• install: keep the user PATH REG_EXPAND_SZ on Windows (#523) (a12473b)
• judge: explain decision-model refusals, follow the attempt's judge in captures and scale Gemini clicks (#544) (d2f9bc9)
• language-runner: keep a remote body's checks in its attempt record (#507) (9f094e0)
• python: name the wire types' schema sources with POSIX paths on every host (#528) (eee7c3d)
• ui: count a study as not gated, never pass (#541) (c2442eb)
• usage: count reasoning tokens once, as part of output (#525) (4029d3d)

v0.13.0

0.13.0 (2026-10-23)

Features

• judge: call Anthropic and OpenAI models directly (#481) (06529de)
• skills: add setup skill (#464) (a8398c9)
• workspace: add a PR-lift recipe and flag compare arms that run one tree (#485) (10c1f05)

Bug Fixes

• harness: count Antigravity tokens per turn, not agy's running total (#495) (3f02ff0)
• harness: count Codex tokens per turn, not the thread's running total (#496) (b032421)
• harness: count Cursor cache reads once in token usage (#487) (f087172)
• harness: map the tools and subagent calls the nightly contract caught (#469) (246a78d)
• judge: stop sending temperature to Anthropic models (#475) (7f1dcba)
• run: name the planned sandbox in the run header and record (#484) (0468d63)
• sandbox: forward absolute-form https requests through the egress gateway (#479) (17ad0de)
• sandbox: refuse private addresses under allow-all and harden the gateway (#490) (93ac6a3)
• sandbox: run privileged execs without --privileged (#489) (08240ac)
• sandbox: stop empty file writes hanging docker exec, and fail stuck fixtures fast (#493) (8682621)
• stats: use one minimum of 4 attempts across views (#477) (b55cff9)

v0.12.0

0.12.0 (2026-10-22)

⚠ BREAKING CHANGES

• insights: classify and explain each eval at the end of reeval run (#451)
• cli: scaffold an environment, add --bare and a docker check to init (#444)
• compare: print reeval compare as the catch-regressions layout (#425)

Features

• cli: scaffold an environment, add --bare and a docker check to init (#444) (3f97710)
• compare: print reeval compare as the catch-regressions layout (#425) (185eb94)
• environment: plugin builds (#437) (7f9e255)
• environment: plugins input (#435) (9186e05)
• harness: plugins on Claude Code and Grok Build (#439) (687e5a8)
• init: set analysis.model in the generated config, warn in doctor when it's unset, and document insights and remarks (#459) (f4e5d1f)
• insights: classify and explain each eval at the end of reeval run (#451) (5c21e46)
• insights: write a recorded run's insights with reeval analyze --insights, and print them in reeval show (#426) (cfcd501)
• mcp: time server start-up (#434) (1361bae)
• plugins: plugin.marketplace, and plugins on Codex, Goose, Antigravity, Cursor, OpenCode, pi and fx (#455) (f0d9a9a)
• remarks: end reeval run with deterministic remarks on the evals themselves (#450) (7aa8b27)
• remarks: raise model remarks behind kind and quote gates, with the eval rubric, a flaky insight tier and a one-line skip (#460) (bd4ea8a)
• test: enforce plan-case tier for eval options in CI (#376) (d0dade0)

Bug Fixes

• claude-code: turn off AskUserQuestion so the next turn can start (#466) (25247be)
• cursor: default fast to false so unpinned Cursor runs are standard-priced (#467) (0f3be6a)
• history: serialize concurrent history.db syncs with begin immediate (#452) (74ae40b)
• insights: refuse a headline that says all of a single result (#465) (f87d643)
• release: open a release PR per package so a reeval-only release tags (#432) (3464ff6)
• remarks: count a remark as seen only in a run that printed it, and find the default branch in ci (#457) (04d117c)
• results: fence summary.md blocks past any backticks they carry (#454) (99fa22b)
• sandbox: keep the diff's snapshot unreadable and free of pre-seed content (#438) (96129ee)
• sandbox: run every exec with umask 022 (#456) (cebf376)
• stats: mcnemar overflow, family rank ties, effort test direction and tied floor (#463) (7838776)
• view: make SuiteResult agree with the exit code and list every check (#430) (243ad7b)