v0.26.0

0.26.0 (2026-11-13)

⚠ BREAKING CHANGES

• cli: require a linked project for a cloud run and check its secrets first (#972)

Features

• api: add a typed client for the cloud platform's API (#975) (eb91fce)
• cli: require a linked project for a cloud run and check its secrets first (#972) (b238ebb)
• cloud: cancel a cloud run with reeval cloud run cancel, and exit 4 when the platform stopped it (#933) (747055a)
• run: resume a bundle's run with --bundle and --resume (#932) (a6f44f2)
• trace: export spans, logs and host metrics over OTLP during the run (#974) (c95599a)

Bug Fixes

• fx: move the relay port below the linux ephemeral range (#969) (1f7ae3b)
• login: print the cloud URL without a trailing slash and read sign-ins stored under it (#971) (acfb468)
• login: warn about each sign-in login and logout couldn't revoke (#973) (8515eee)
• resume: write retries and resume stubs atomically, and ignore this process's own pid (#931) (5a1dfcf)
• sandbox: keep fixed loopback ports out of the ephemeral range (#967) (0345795)

v0.25.0

0.25.0 (2026-11-13)

⚠ BREAKING CHANGES

• studio: serve the run report in place of the run and attempt views (#942)
• studio: reeval run --artifacts is removed; set history.artifacts, or pass reeval record --artifacts or reeval cloud push --artifacts.
• studio: serve a typed studio API on loopback to a paired studio page (#934)
• history: add composable RunReport summary.md (#935)

Features

• bundle: declare vm-layout in the bundle manifest (#962) (cf784d8)
• cli: add reeval link to link a project to a cloud project (#926) (e526316)
• cloud: push a recorded run to reeval cloud's studio (#936) (f5d8350)
• cloud: push saved analyses with a run (#959) (ddf0607)
• events: emit a heartbeat with the attempts in progress (#963) (1fe9f69)
• history: add composable RunReport summary.md (#935) (c58b3ef)
• history: record a trace of what ran where in spans.jsonl (#966) (46feb4d)
• report: add each attempt's transcript facts to the run report (#958) (2666d47)
• sandbox: add reeval/internal/vm-api, adapters over the Boat and Daytona SDKs (#943) (f26ada3)
• sandbox: stop a cloud run's starts when the service says the run can't go on (#941) (fd66b4e)
• sandbox: take cloud()'s layout from the service and give back reclaimed machines (#940) (07b7bcb)
• sandbox: wait on cloud() starts until the service ends the run (#939) (085c3df)
• studio: serve a typed studio API on loopback to a paired studio page (#934) (cee5ae8)
• studio: serve the run report in place of the run and attempt views (#942) (d105ba7)
• studio: share a recorded run from the cli and write a push archive to a file (#937) (365874e)
• ui: report when an attempt's sandbox is up (#938) (1b778a9)
• view: derive a run's scope for like-with-like history and trends (#945) (eaba735)

Bug Fixes

• compare: treat a crashed attempt the engine passed as infra_error in questions (#952) (24cef15)
• fx: bump fx to 0.0.16, refuse mcp.sse and pass the sandbox env to stdio MCP servers (#948) (e07599e)
• login: key the platform sign-in by its bare origin (#965) (a3a33f6)
• sandbox: fold a reading too close to the last into its sample, so avgCores never tops peakCores (#957) (ed30676)
• sandbox: record no status for a brokered request the sandbox canceled (#955) (d0fd4f7)
• sandbox: remove the containers a killed reeval left running (#964) (f2cb1e9)

v0.24.0

0.24.0 (2026-11-10)

⚠ BREAKING CHANGES

• cli: download a cloud run's files as a run here leaves them (#915)
• cli: rename the hosted run option to cloud (#908)

Features

• cli: add reeval cloud secrets and reeval cloud provider (#911) (f5717d3)
• cli: add reeval login, logout and whoami for the cloud platform (#910) (392bd05)
• cli: default an unset REEVAL_CLOUD_URL (#925) (475affc)
• cli: download a cloud run's files as a run here leaves them (#915) (5e5b1ed)
• cli: download a cloud run's files with --download (#913) (fd8fcc7)
• cli: name where a recorded run landed in the run-end event (#914) (c0298d1)
• cli: rename the hosted run option to cloud (#908) (fdda569)
• history: add history.archive, which records each run as a zip beside its record (#905) (007f7a5)
• history: add history.debugArchive, which zips a run's local debug data beside it (#906) (49a0435)
• secrets: honor REEVAL_SECRETS_FILE modes and record secretWarnings (#912) (7f59fdd)

Bug Fixes

• cli: accept typesafe-ai in reeval cloud provider (#916) (67d334c)

v0.23.0

0.23.0 (2026-11-08)

Features

• cli: list, clean and build boat() and daytona() image snapshots (#901) (62f01a2)
• cli: run a project on the hosted platform with reeval run (#899) (9f44441)
• history: add history archives, a zip of one recorded run with a hashed manifest (#904) (e64afd6)
• sandbox: add a daytona() sandbox (#883) (d7d5dab)
• sandbox: add an internal VM sandbox (a7862bd)
• sandbox: build boat image recipes into saved snapshots (#898) (3f02fc6)
• sandbox: build daytona() images from image, from, dockerfile and install (#896) (d799650)
• sandbox: record the VM an attempt ran on and label VMs from REEVAL_SANDBOX_LABELS (#888) (cd589bf)
• sandbox: run boat() attempts in a container of an image recipe (#900) (4037242)
• sandbox: run the docker CLI on a remote machine over SSH (#895) (7e38f85)
• sandbox: share an account's limits between its machine pools (#902) (baf4529)

Bug Fixes

• attach: print a run's paths as the viewer's project's, not the runner's directory (#887) (4314be2)
• bundle: record the bundled project's git state and author on a run from a bundle (#885) (e792b80)
• capture: allocate c.serve's PORT outside the ephemeral port range (#894) (a77ef4d)
• sandbox: create the resource sampler's observer before teardown can race it (#892) (fabb3e1)
• sandbox: give a daytona() sandbox user the login user's node and npm (#886) (1df403d)
• sandbox: read a daytona() sandbox's resources from its cgroup (#889) (76d2005)

v0.22.0

0.22.0 (2026-11-07)

⚠ BREAKING CHANGES

• cli: follow a run over a WebSocket in reeval attach (#877)
• sandbox: key image tags on content and push or pull them through a registry (#860)

Features

• cli: bundle a run and run it from the bundle (#856) (ba52e47)
• cli: draw reeval run's interactive list in reeval attach on a terminal (#862) (e1555ea)
• cli: follow a run over a WebSocket in reeval attach (#877) (5c41ee8)
• cli: stream run events to ndjson and follow them with reeval attach (#857) (2ddee33)
• cli: write heartbeats to reeval run's event stream and show a stalled or lost run in reeval attach (#863) (e02d954)
• gateway: send vercel() calls to AI_GATEWAY_BASE_URL (#859) (5a5d3d0)
• plan: list the host files a run reads in --dry-run --json (#855) (8ac7efe)
• sandbox: key image tags on content and push or pull them through a registry (#860) (5cc52bc)
• sandbox: pool remote machines across a run, with reuse and attemptsPerMachine (#876) (f1b333c)
• sandbox: reach a boat() VM at its own address first, then the public endpoint (#875) (78fdb80)
• sandbox: record each attempt's machine and resource use (#878) (46a1491)
• sandbox: run every harness in boat() (#861) (1939454)
• sandbox: sample resources out of the agent's reach and record throttling, memory.peak and p95 cores (#879) (fb772ce)

Bug Fixes

• cli: treat a Ctrl-C that races attach's cancel as the same press (#865) (cf3b07e)
• expect: keep a caught expect.require's chai failure instead of a TypeError (#871) (305e92f)
• redaction: mask secrets in expect failures before the message cuts them (#867) (7ad30d2)
• sandbox: kill an aborted command's process group in tart and boat machines (#874) (d6a4a3e)
• ui: mask what the interactive list draws, and progress at its source (#866) (820aceb)

Performance Improvements

• sandbox: install the harness once per boat() machine and copy it into each slot (#880) (375df9a)

v0.21.0

0.21.0 (2026-11-06)

⚠ BREAKING CHANGES

• harness: run pi and goose on any provider they key (#829)
• history: configure history.store, per profile (#843)
• history: default history.artifacts to failures and mark recorded runs generated (#801)

Features

• egress: separate setup and agent egress policies (#799) (4d920b8)
• exec: add c.exec(cmd, { env }) for per-command variables (#825) (60b58ad)
• harness: recover antigravity reasoning from agy's stored conversation (#833) (dab80e9)
• harness: run pi and goose on any provider they key (#829) (b04eff6)
• history: configure history.store, per profile (#843) (4060a19)
• history: default history.artifacts to failures and mark recorded runs generated (#801) (2b1711f)
• history: read history from a store (#846) (bc86a47)
• history: record runs to an S3-compatible store (#845) (9c56200)
• history: store and show cost per turn, attempt, cell and run (#835) (026da8f)
• history: time each turn's model calls, tools and waiting (#831) (9c84594)
• history: upload debug data on request (#848) (f948a64)
• matchers: match the program a shell command ran with toHaveRun({ program, args }) (#823) (5bc864f)
• sandbox: bake harness images for tart and run pi and codex there (#795) (452dce6)
• sandbox: run antigravity, goose, fx and grok build in tart (#813) (fae373b)
• sandbox: run claude code and opencode in tart (#798) (e92c609)
• sandbox: run tart() vms on remote macs over ssh (#809) (6977d45)
• secrets: warn about plain env strings that look like credentials (#800) (317b282)

Bug Fixes

• assert: recognize every harness's skill use in toHaveCalledSkill and skill_reads (#824) (c88dd24)
• batch environment files, and correct dry-run egress, progress, init and doctor output (#811) (ea54f2c)
• cli: keep resumable staging in the sweep and never sweep on --dry-run (#822) (6d764f9)
• egress: fall back to every checked address when connecting upstream (#810) (48fae1d)
• harness: keep an aborted turn's streamed events and stored usage (#840) (929464d)
• harness: record a turn the model provider stopped as an infra error (#827) (87ab8cd)
• harness: record fx's cache reads and reasoning tokens on Chat Completions providers (#839) (a8ab9a5)
• harness: record goose's tokens from its usage_ledger after each turn (#838) (4d3a0af)
• harness: record large tool output that cursor and antigravity save to files (#842) (c3eabad)
• harness: record reasoning tokens and text on claude code, codex and opencode (#841) (bd59533)
• harness: retry cursor's catalog and transient start failures, and keep one start's abort from failing another's (#850) (4b65647)
• harness: run claudeCode() on a claude setup-token subscription token (#814) (4f79903)
• history: continue evaluate.each history after the move to cases (#794) (c1d7fa7)
• history: count skill use of skills reeval didn't place (#834) (fcd974b)
• history: keep a fresh pending marker, retry Bun's network errors, and check the CI store without pulling (#853) (df6bc92)
• init: derive the analysis key from the analysis model's provider (#847) (e807211)
• judge: record a judge call the provider never answered as an infra error (#821) (ddb5fed)
• opencode: record the turn's uncached input instead of 0 (#837) (0223e94)
• sandbox: close the raw test connections podman's slirp4netns keeps open (#816) (745bf5b)
• sandbox: delete a stopped run's boat vms on --resume (#828) (8f0d903)
• sandbox: keep ssh control socket paths under the socket length limit (#820) (68dd499)
• ui: print the served-by upstream lines in reeval run's status list (#830) (3a383ae)

v0.20.0

0.20.0 (2026-11-03)

Features

• tart: run the Cursor harness in tart() on a harness session in the VM (#790) (b37594a)

v0.19.0

0.19.0 (2026-11-03)

⚠ BREAKING CHANGES

• sandbox: pull tart() images in reeval image build, never in a run (#784)
• history: make an eval with cases one eval in records, views and the plan (#759)
• cli: add reeval show --trace, a run's timeline for Perfetto, and drop run's --trace alias (#777)
• authoring API groundwork (plural axes, compare.control, cases, where, --trait) (#745)

Features

• authoring API groundwork (plural axes, compare.control, cases, where, --trait) (#745) (008f984)
• cli: add reeval show --trace, a run's timeline for Perfetto, and drop run's --trace alias (#777) (4fdbb76)
• cli: rerun a finished run's infra errors with reeval run --retry-errors (#785) (c93714b)
• history: make an eval with cases one eval in records, views and the plan (#759) (c7f6608)
• sandbox: add a tart() sandbox kind that plans and prepares macOS VMs, at most two at once (#774) (988a590)
• sandbox: add the boat sandbox provider (#773) (3ff6bff)
• sandbox: pull tart() images in reeval image build, never in a run (#784) (22d8a9f)
• sandbox: run each tart() attempt in a cloned macOS VM over tart exec, with SSH as the fallback (#775) (39909ac)
• ui: show an eval's case table live in the interactive list, five cases then all (#783) (625158e)
• ui: spin while waiting, print failed checks as fails and tighten the compare block (#770) (d3a0961)

Bug Fixes

• cli: keep a staged rerun and finish a killed --retry-errors on --resume (#788) (0dafda1)
• sandbox: give boat attempts the start-up allowance a vm boot needs (#789) (b03708d)
• ui: measure the prepare spinner in columns and finish the run output follow-ups (#781) (2808b8a)

v0.18.0

0.18.0 (2026-11-02)

⚠ BREAKING CHANGES

• ui: print reeval run's results as a per-eval status list (#765)
• results: show a lift's direction at any number of runs (#762)

Features

• judge: show c.observe values to judges and questions, in compare and the studio (#743) (99af246)
• record non-gating attempt values with c.observe (#741) (cdb582f)
• results: compare each variant with its baseline on every measure (#764) (a82029b)
• results: show a lift's direction at any number of runs (#762) (74eb5e9)
• ui: print reeval run's results as a per-eval status list (#765) (b9af22c)

Bug Fixes

• analysis: default maxOutputTokens to 32000 and name the option on a cut-off (#747) (8ed883b)
• assert: back toHaveEdited with the files each turn changed (#763) (9e05dcf)
• capture: give the date format hint only while a date or time field has focus (#744) (862741a)
• capture: make vision journeys hold up on scrolled pages, frames and late content (#740) (fd66d32)
• ci: drop retired Claude 4.0 ids and fail docs:check on retired Claude models (#754) (321fcd1)
• ci: retire claude-sonnet-4-5 and refuse claude code's sonnet alias on vertex (#761) (c90e09d)
• ci: run claude code's contract rows on claude-haiku-5-5 and the plan cases on claude-sonnet-5-5 (#757) (e432bd5)
• cli: keep the attempts a filtered resume leaves out, and the run incomplete (#753) (eb077b3)
• cli: make --resume plan with the run's own --runs, profile, filters and concurrency (#749) (416df31)
• docs: skip changelog and captured data in the retired-model guard and catch Bedrock and claude-4-sonnet forms (#755) (29c0aef)
• evaluate: mask secrets in eval, group and todo names, and refuse names that mask alike (#752) (d491f0d)
• history: hash evaluate.each case data with its secrets masked (#756) (d711952)
• history: mark declared skips in the record so a filtered resume counts them as done (#758) (0b82b71)
• observe: mask observation names and refuse two that mask the same (#748) (8f414d8)
• sandbox: turn off npm's and pnpm's update checks in the container (#760) (f846a1d)
• serve: refuse a given port that was already in use before c.serve started (#746) (36c1707)

v0.17.2

0.17.2 (2026-10-30)

Bug Fixes

• cli: read a passed-on Ctrl-C as one press whichever signal arrives first (#738) (3c02c33)
• harness: record unmeasured reasoning time as null and count subagent edits (#736) (0ee5467)