Repository navigation
v0.4.0
Minor Changes
-
Measured review rates instead of impressions. A marked
corpus-*bundle kind
carries a checked-in case manifest with ground-truth bugs and provenance;
calibrategrades a model's findings against that ground truth offline and
reports precision, recall, and false-positive rate, andtrendaggregates the
same numbers per model and panel. Ordinarylocal-*reviews keep refusing the
live join. Ships a seeded starter corpus underbench/corpus/and
docs/benchmark.md, an operator runbook for real PRs, real agents, cost
estimates, and publishing measured rates. -
Experimental MCP server.
verdict mcpserves the review verbs to an MCP host
over stdio (newline-delimited JSON-RPC 2.0): no daemon, no HTTP, no new
dependency, sharing the host's credentials and exiting when stdin closes.
Tool schemas reflect the CLI request types, results are the--json
envelope'sdata, and typed refusals returnisErrorwith code, message, and
hint intact;--read-onlyexposes only the non-writing verbs, whilepostis
available in normal mode and in--operatormode.verdict mcp config --agent claude|cursor|codexprints the exact host entry, and--writemerges it
idempotently into the documented config (.mcp.json,.cursor/mcp.json, or
$CODEX_HOME/config.toml), backing the old file up to<path>.verdict.bak
and refusing unparseable or foreign entries instead of rewriting them.--json
prints one envelope, never prompts, and never starts the server;verdict --helpnow lists the MCP entry so the surface is discoverable. -
verdictis one static binary — no checkout, nojq, no shell script. Install
it with the verifiedinstall.sh(macOS/Linux) orinstall.ps1(Windows) into
~/.verdict/bin, the Homebrew tap, or the npm wrapper; both installers verify
the cosign signature and the archive SHA-256, smoke-test the installed binary
from its final path, and addPATHwith one marked line
(VERDICT_NO_MODIFY_PATH=1opts out). In CI the newsetup-verdictaction
installs a pinned, verified copy on Ubuntu, macOS, and Windows runners without
touching shell rc files.verdict upgradeupdates whichever channel installed
it — a verified self-replace for installer installs, the owning package
manager's global update for npm and Homebrew — andverdict uninstallremoves
only installer-owned files; both refuse, naming the exact command, when the
install belongs to something else.verdict skillprints or installs the
payload embedded in the binary (SKILL.md,prompts/,schemas/, and the
scripts/verdictshim), so the served skill can never skew from the binary.Upgrading from 0.3.x: the old script install is replaced, not patched.
scripts/review.shis gone — every verb isverdict <verb>with the same
flags and--jsonenvelopes. The old symlink-onlyinstall.shbecame
install-skill.sh(checkout-only), and the copy-basedinstallverb is gone,
so reinstall the skill withverdict skill installornpx skills add. Docs
and scripts that call the old script path must be updated. Start at the
rewritten quickstart, the new glossary of review terms, the CI recipe in
docs/ci-recipe.md, andverdict --help, which now names each verb's
provider scope. -
Posting is harder to get wrong and harder to do twice.
postrefuses a second
review on the same reviewed SHA before any network read, naming the exact
fetch --base-shaadvance; it also refuses duplicates found in the bundle's
own fetched review list, so a fresh checkout, a CI re-run, or another machine
cannot repost what the PR already has —--forcestill overrides, and the
refusal names the re-fetch needed when the earlier review was deleted. Points
outside the diff hunks post under "Unanchored observations" instead of as
inline anchors GitHub would reject. A body past the 65,536-character hosting
limit splits into follow-up COMMENT reviews — first review keeps the footer,
later parts carry a continuation header, and a failed follow-up refuses typed
naming the review that already landed — instead of being trimmed. A 403/429
carrying a rate-limit signal retries within bounds, honoringretry-afteror
the reset marker and never publishing twice; when the last permitted attempt
is still retryable, the refusal carriesretry_atin RFC 3339 so an agent can
wait for that instant.A stale-fetch downgrade note is now followed by a blank line, like the drift note, instead of a stray literal
\nin the posted review body. -
The PR conversation is a local, schema'd record instead of something each verb
re-reads from the host. Every fetch writes.review/<pr>/threads.json(schema
published) holding each thread's host id, diff anchor, resolved and outdated
state read from the host's own fields, root author kind, and the comments with
authors, timestamps, and capped bodies; GitHub comes from GraphQL
reviewThreads(REST carries no resolution), and GitLab, GitCafe, Forgejo,
Bitbucket, and Azure fill the same shape, paging rather than truncating.
verdict threads <owner/repo> <pr> [--provider …] [--json]re-reads just that
surface and rewrites only that file, so re-review, address-reviews, readiness,
and calibration can join prior feedback on any of the six hosts. Posted inline
comments now carry a hidden final-line<!-- verdict-finding sig=… -->marker,
so a new comment links back to the finding that produced it while older
comments keep reading as unlinked. -
Published machine contracts beside the CLI.
schemas/ships JSON Schemas for
the review input, findings store, cost and usage records, PR threads, and the
--jsonenvelope with per-verbdataprofiles, included in the npm tarball;
schemas/validate.shchecks documents against them offline (nojq; exit 0
valid, 1 rejected, 2 bad usage), and the drift rule is additive within schema
version 1. An experimental TypeScript client
(@stainless-code/verdict/client) wraps each verb with envelope types
generated from those schemas and refusal codes generated from the CLI, with no
runtime dependencies. The experimental Go packagepkg/verdictembeds the
same operations in-process through a context-first client with typed requests
and responses and a*RefusalErrorforerrors.As; its API may still change
before 1.0. A new "Build a client" guide documents the--jsonprotocol —
envelope keys, exit and refusal codes, versioning, binary resolution,
timeouts, output caps, and when MCP beats spawning a process — with a
dependency-free Python client generated from the schemas and an offline smoke
test. -
The readiness gate can say what it knows.
report --checkand
report --completenessgain a live-thread dimension: open threads the local
store does not know about fail the gate, and unreachable lookups warn and stay
neutral. New--live auto|strict|skippicks the posture —autokeeps that
fail-open default,strictfails closed when the host cannot be reached, and
skippays zero network and gates on the local dimensions only, saying so in
the output and the--jsonwarnings.postand the gate now name the same
advance for head drift, and a binding APPROVE on a known-stale fetch
downgrades to COMMENT instead of approving past the moved head. -
Chunked split-range reviews. Very large PRs are reviewable again: with
--chunk(or.review/chunk) an over-cap diff is sliced on file boundaries
into numbered chunks with a versioned manifest, the agent reviews one chunk at
a time, and the results synthesize into one verdict with a summed Basis line.
postandrecordrefuse coverage that claims to be complete but is attested
partial, and warn loudly when a chunked review is unattested.fetch-local
supports the same flow. -
verifyrefuses before it runs. Destructive, networked, credential-reading,
and repo-escaping commands are rejected with a typed error before execution
(over 40 regression shapes), so a hostile or carelessverifyCmdcannot
delete, exfiltrate, or escape the repository. The recorded tail also drops
credentials no environment value names — vendor-prefixed tokens,
AuthorizationandProxy-Authorizationheaders, bareBearervalues, and
URL userinfo — and strips ANSI escapes and other control characters, so a
command cannot rewrite the proof it lands in;.verify.outkeeps the raw
bytes.doctorwarns on excess token scope, and the manual documents an
opt-in hostile-PR sandbox recipe plus the residual risk a string scan cannot
remove.
Patch Changes
-
Cost accounting reads what the operator actually knows.
cost.jsonentries
gain optionalcache_read_input_tokensandcache_creation_input_tokens
(subsets ofinput_tokens; absent ornullreads unknown), and
report --costrolls them into a top-levelcached_input_tokensscalar with
totals unchanged. The spend gate can now price comparable runs from usage
records the operator hands over withVERDICT_USAGE=<file|dir>, in a published
schema (schema, model, tokens, optional USD, source, window), with the
self-reportedcost.jsonas the fallback; a handed record joins by its window
bracketing the run, and a named path that is missing or malformed refuses typed
before the bundle is written. Automatic discovery of other tools' transcripts
is gone — the optional converters live outside the binary in
examples/usage-adapters/. With nothing handed over, output is byte-identical. -
Isolated local reviews.
fetch-local --worktreechecks the review target into
a temporary detached worktree and builds the bundle from that checkout instead
of the live tree, so caller tracing and verification run against real, stable
code without disturbing the working tree. The checkout is always removed
before the verb exits (success, refusal, or cancel); when no checkout can be
created the verb refuses typedworktree-failedand writes nothing. Without
the flag every byte behaves as before. -
Small additive reads.
post.jsonandrecord.jsonnow recordtool_version,
prompts_hash, andmodel, andcalibrateandtrendsurface them so a
mixed-version history reads as mixed.show --jsongains adata.fixblock
beside the unchangeddata.item: full sig, status, severity, deduped files,
locator, suggestion, averify_hintread from the bundle, and a fixed
untrusted-evidence caution — a pure read with no flags, no execution, and no
store writes, and missing history reads as null rather than invented text.
doctorhuman output opens with a run stamp and closes with one line,
Summary: healthy|unhealthy — N ok, M warn, K fail, while--jsonenvelopes
and exits stay byte-identical. Builds with no release metadata now say
0.0.0-unknowneverywhere —--version, the skill payload and receipt, and
the posted or reported footer — instead of the footer sayingunknown. -
Failures a 0.3.1 user could hit are gone.
fetchon GitLab looks up the
project id with the plainowner%2Frepopath — the old request carried a
trailing encoded newline, which could fail the lookup at the first step.
Provider lists are read completely: Forgejo files and reviews, GitLab MR notes,
and Azure change entries and threads page instead of stopping at the first
page, and a page cap, a failed page, or a failed thread read lands an envelope
warning instead of silently under-covering.fetchretries GitHub's transient
"internal error, please try again" on the same three-attempt budgetpost
already had, while auth and validation failures still fail fast. Machine mode
never prompts, even on a TTY:init --jsonandinstall --jsonwithout
--yes/--forcerefuse typedneeds-confirm, write nothing, and print the
hint instead of waiting on aProceed?prompt. Malformed agent input fails
closed withbad-review-input— unparsablereview.json, non-object
comments/triage/sectionsentries, non-string scalars, misshapen
verification claims — and a corrupt findings store now refuses typed
corrupt-storeacrossreport,show,calibrate, andtrendinstead of
crashing mid-render with no envelope, never starting fresh over history.A value flag given without its value now fails as a usage error naming what
it wanted (--provider needs a value (want <name>), exit 2) before any
network call, instead of falling back to a default target. A bare--stash
still meansstash@{0}.Cancellation is clean. SIGTERM now cancels a running verb the way Ctrl-C
does: exit 3 with acanceledenvelope, and child process groups are killed
instead of orphaned. A cancelled verb returns promptly even when a grandchild
process still holds its output pipe. Interruptingverifyrefuses
canceledand records nothing, where it used to record a false
timeout after 300sproof and exit 0. Themcpstdio server takes the same
signal: SIGTERM cancels an in-flight tool call and kills its process tree
instead of orphaning a runningverifyfor the rest of the verify budget.
On Windows,verdict upgradestages its
helper beside the installed binary, so an install directory on another drive
no longer makes the final move fail after the upgrade reported success.
On Windows,verifystarts its command suspended and puts it in its
kill-on-close job object before it runs, so a grandchild spawned in the first
instant can no longer escape a timeout or cancel kill.
threads.jsonalways labels a thread's author kind, including a thread with
no readable comments, as the published schema requires. A cancel that lands
inside a read verb's service (ready,trend,threads,calibrate,
check-tree,doctor,report,next) exits 3 with thecanceledenvelope
like every other verb, where it used to exit 1 as an ordinary refusal.Provider page walks finish what they start. A GitLab merge request whose diff
list spans host pages (glab api --paginateconcatenates one JSON array per
page) now yields every changed file instead of aPR.jsonwith no files at
all, and a merged Bitbucket pull request (state: MERGED) folds tomerged
rather thanunknown. A GitCafe review/comment cursor walk that stops early — a
failed page, an unreadable page, or the 20-page cap — keeps the pages it already
read, warns, and marksthreads.jsontruncated: trueinstead of quietly
under-covering the conversation; Forgejo's review list does the same at the
provider cap. Hosts with no failures read exactly as before.
A failed or unreadable Bitbucket diffstat/comments page, a GitLab changed-file
payload that is not a diff list, and a failed Azure iteration read no longer
pass as an empty list: each warns, and the Bitbucket threads feed is marked
truncated: true.Report artifacts survive a kill.
report --output,report --sarif, and the
MCP report artifact are staged beside the target and renamed into place, so a
crash mid-write can no longer leave a truncated report behind. A symlinked
--outputpath keeps the link and replaces only the file it names, an existing
file keeps its own mode (a new one is still owner-only), and a device like
/dev/stdoutstill receives the bytes directly. A failed write refuses
report-failedand leaves the previous report intact.Numbers in machine output render the way jq 1.8 renders them. A literal such as
9.223372036854775808e18now prints as9223372036854775808on every
architecture, where arm64 used to saturate at9223372036854775807and amd64
wrap to-9223372036854775808.verdict mcp installno longer corrupts a Codexconfig.tomlthat uses a TOML
multi-line string: a[mcp_servers.verdict]header spelled inside a'''or
"""literal used to be merged into as if it were real, and the merge now
refusesmcp-config-unmergeable(file untouched, add the table by hand)
instead of guessing. -
Reviews of other repositories stop carrying this repository's own conventions.
The shared system prompt now defers branch, commit-message, formatting, and
style rules to the target repo'sAGENTS.md/CLAUDE.md/CONTRIBUTING, with
verdict's own rules staying in itsAGENTS.md, so reviews of this repo keep
them. The fix flow opens with the untrusted-data preamble, the abort-if-not-
introduced pin is stated where the flow can follow it, and the manual records
the machine-modes-never-prompt doctrine and a combined
report --check --completenesspre-push probe.--show-promptswraps every
untrusted bundle section in a per-run nonce fence named in a trusted prologue,
so a forged close smuggled in a diff stays evidence instead of ending the
section; thread files stay off the initial-review reading list by design.
Missing trusted prompt files no longer vanish silently: a missing
prompts/system.mdrefuses typed, while a missingSKILL.md,
prompts/review.md, orprompts/diagrams.mdwarns and proceeds; missing
bundle files still skip silently. -
Credential and untrusted-input hardening. Every credential-bearing provider
read is bound to the configured origin:fetchrefuses a Bitbucket pagination
next, a Forgejo metadatadiff_url, or an Azure metadataremoteUrlwhose
scheme and host differ from the API base, instead of sending the token to that
host, and Bitbucket paging stops at a page cap or a repeated cursor with a
warning that the bundle may be incomplete. Command-stderr sidecars written by
fetchandpost(.diff.err,.post.err,.recheck.err, provider.err)
are now owner-only0600, since they can carry token-adjacent forge
diagnostics. Untrusted input is capped:recordandpostrefuse an oversized
agent file while reading it instead of after allocating the whole document,
verifyand the fetch runners cap captured child output (a capped proof
records a truncation marker instead of growing the process), and every store
reader rejects oversized documents while keeping its own refusal code. A review
anchor, knowledge-base counter, override threshold, adopt anchor, verify
timeout, or read number that does not fit in an integer now takes its typed
refusal instead of wrapping into a nonsense negative span or count.The
verifygate now refuses a command it cannot fully scan. A destructive
tool name hidden inside$(…)substitutions nested deeper than the gate's
scan bound, or inside a substitution body glued directly to a word (for
example$( (echo r))m -rf dir), was previously treated as clean. Both now
refuse before anything runs. A pull request file name containing a tab or
newline can no longer forge or shift rows inhunks.tsv, and the ordered
JSON reader rejects trailing bytes after the document ({"a":1}}) instead of
silently ignoring them.