Measured on 49 benchmark files in full agent sessions with Claude, Copilot and Codex:
this release finds as many bugs as an agent reading the files itself, or more, with
32–69% fewer tokens, and matches v0.8.2's accuracy with Claude at 78% fewer tokens.
With Copilot, v0.8.2 found two more bugs over two sessions at 2.5 times the tokens.
See the results.
Added
check_typesreturns selected JavaScript/TypeScript files' compiler diagnostics
without model calls. It preserves configured project declarations and options,
reports incomplete context and limits, and bundles TypeScript's standard libraries.
Compiler diagnostics are separate from model confidence and runtime predictions.- Imported definitions now include CommonJS:
require()bindings and exports from
module.exports/exports. When a required name provably is not exported (the
module's exports are one complete object literal), the model is told so; in any
less certain shape nothing is claimed. Not yet benchmarked. - Replies to
predict_failuresincludeproviderVersion, the CLI version that
produced the verdicts, andcached: truefor a reused verdict (below). npm run bench:canary -- --provider=<id>checks an installed CLI against 12
held-out benchmark files in about a minute and reports PASS or DRIFT.
Changed
- Small files share model calls: up to eight per call, with a bounded prompt, each
keeping its own verdict and line validation. Large files still run alone. A file
whose group verdict names a defect with a score under 0.80, or that the group
calls clean although it reads, awaits and writes back shared state, is reviewed
again on its own, and that verdict replaces the group's. Missing or malformed
group verdicts are unavailable, never clean. - Scores are anchored to a trigger the shown code permits and a result the shown
code contradicts, and the expected behavior must come from the code, not from the
model's own assumptions. uncertainresults are no longer something to ignore: replies containing one add
check, asking the calling agent to read the cited lines and confirm or dismiss it.- Re-reviewing a set after editing one file no longer pays for the unchanged ones.
Within a server session, identical review input returns the earlier verdict with
cached: true; failed and unavailable verdicts are never reused. - After two failed provider calls in a row, the remaining files fail at once with
the reason instead of each waiting out its timeout. Re-checks start as soon as
their group finishes, and files are read and parsed in parallel. - Claude reviews no longer make a hidden side call that sent every prompt uncached,
and no longer load your skills, MCP servers or session files. concurrencycounts model calls in flight, each covering up to eight files.bench/RESULTS.mdsummarizes the current comparison; older reports remain at the
release tags.npm run benchis replaced bynpm run bench:canary.
Security
- Only JavaScript/TypeScript source files are sent to the model. A path to a
.env
or key file is refused before any provider call, imports resolve only to source
files, and one call accepts at most 100 files. - Each review call starts the CLI in a new, empty temporary folder, deleted
afterwards. In the project folder, all three CLIs loaded itsCLAUDE.md/AGENTS.md
into every review, costing tokens and letting the repository under review instruct
its reviewer. See SECURITY.md. - Codex reviews turn off Codex's agent tools (shell, patching, web, browser, computer
use, plugins, sub-agents, image generation). With its defaults, asked to read a
file, the reviewing Codex called its shell. Each Codex call now sends about 9k more
uncached tokens, because its shared prompt cache no longer applies. - Copilot reviews expose no tools to the model, including MCP tools from your own
configuration. - MCP tools declare themselves read-only and non-destructive, which also lets Codex
call them with approvals off; v0.8.2 could not run there.
Fixed
- Brackets in prose before a model's verdict (
checked [x],line [12]) no longer
make the verdict unavailable. - An explicit
unknownverdict stays unavailable instead of becoming a clean result.