Skip to content

v0.7.0

Latest

Choose a tag to compare

@github-actions github-actions released this 16 Nov 14:45
· 81 commits to main since this release
b4c3634

Minor Changes

  • What's new in 0.7.0

    Verdict 0.7.0 is a breadth release. It adds a local code index, a read-only serve surface with a browser
    UI, and a backtest harness that mines past bug fixes from your own history, replays them against your
    agent, and scores the result. Review output gains an auditable provenance receipt, diff-coverage
    attestation, advisory risk scoring, and an optional validation pass that floors each finding on the
    evidence behind it, while the CI gate now checks coverage by default. Posting grows a secret guard, honest
    per-host capability reporting, and a summary-comment option, GitLab and Forgejo join GitHub as full publish
    targets, and lens packs, declared recipes, and reported verification make the review pipeline itself data.
    Thirteen pre-1.0 changes alter behavior — each names the migration step below — and the known limits close
    the notes.

  • New commands and flags

    • pr-wait — wait for a PR to merge, close, change checks, or change review state, with a bounded timeout and typed timeout data. Bounded foreground waits
    • ingest — import SARIF 2.1.0 results into a bundle's findings store, sharing one dedupe and suppression path with model reviews. Static-analysis ingestion
    • threads resolve — close the review threads whose findings your store marks addressed, one receipt per run, with --dry-run and typed refusals for threads verdict did not author. Guarded submission
    • risk — rank every local bundle into an advisory triage queue by band and 0–100 score, without a host call or an index build. Triage risk queue
    • report --html — export the six dashboard views as a static site, published through staging and backup siblings with rollback and symlink refusals. Static dashboard export
    • run --agent — run your own agent CLI against an existing bundle, assert exactly one fresh review.json, then hand off to the unchanged record/post validation. Your agent, launched
    • panel — plan independent verifier dispatches locally; keep-biased results enrich findings and summaries without voting or dropping anything. Panel verification
    • Recipes — .verdict/recipes.json declares a lens set, synthesizer hint, and budget as strict versioned data, read at the PR base and hashed into the bundle. Recipes
    • skill show, skill status, skill uninstall — inspect one payload file byte-identically, gate on drift with an exit code, and remove only the files a receipt owns. Install routes
    • post --summary comment — maintain one separate PR-level summary comment, updated in place across all three publish hosts; the default off keeps output byte-identical. Guarded submission
    • fix --all — the batch form of the bounded fix loop: every open inline finding at or above --min-severity, one pass per sig under the same ceilings, per-sig outcomes in data.batch, and a fix.patch written beside the ledger. Fix loops
    • fix --selected — address exactly the sigs a human 👍 selected on the root verdict comment, read through one shared reaction reader; a selection file is written only when something was reacted to. Fix loops
    • handoff — one markdown projection of the open inline findings fix would take, selecting through the same selector, with --json, --format md, and repeatable --paths; read-only, with no ledger and no host call. Handoff
    • run --print --stage <n> — render one recipe stage at a time, so CI can dispatch the four stages under different models. Recipes
    • backtest reverts — record the reverts git already knows about into a sorted, byte-stable .verdict/reverts.jsonl, with .verdict/reverts.md as the fenced reading surface. Reverts
    • backtest mine — mine SZZ candidates from a local clone with zero model calls and zero network, writing a timestamp-free mine.json. Backtest
    • backtest replay — replay mined candidates against your own agent in a throwaway worktree per candidate, bounded by review count, projected budget, and timeout. Replay
    • backtest score — grade a replay's findings against the candidate bug lines and report a catch rate, off-target findings per PR, and cost per review as a pure read. Score
    • learn --incident — attach one revert or incident to a recorded bundle in a sorted, idempotent .verdict/incidents.jsonl ledger. Knowledge base
  • Local code index

    • verdict index — build a local Go index with the standard library, stored by tree SHA so an empty commit reuses a tree and a rename parses nothing. Code index
    • Graph queries — index tests-for, index co-changed, and index owners answer candidate tests, co-change partners, and the CODEOWNERS rule at a tree. Code index
    • Three more languages — the index now reads TypeScript, JavaScript, and Python through two dependency-free lexical scanners, with no tree-sitter and no cgo. Code index
    • index blast-radius — one change's reach: callers to depth 2 plus candidate tests, co-change partners, and owners for the changed paths. Code index
    • Transitive tests — tests-for walks the reverse import graph up to --depth hops and labels tests it reached through imports. Code index
    • fetch --index — hand an index you already built to fetch or fetch-local and the bundle gains an impact artifact and a fenced ## impact section. Impact hand-off
    • Missing-index hints — index blast-radius and fetch now name the revision and tree an absent index is about, with the exact shell-quoted build step that answers for it. Code index
    • Scanner fixes — four corpus bugs are fixed: a transitive test no longer disappears, relative imports resolve, a Python attribute call is no longer misattributed, and a Go selector outside its package is counted unresolved. Code index
    • One impact budget — the impact section's byte budget is shared across symbols, callers, and tests, each list carrying its own omitted count. Code index
  • Review quality and provenance

    • Explicit context — fetch --context and fetch-local --context attach named context files, and findings cite them. Explicit context
    • Coverage basis — every coverage read now names what its percentage counts: full hunk spans, both sides and context, not just changed lines. Diff coverage
    • Coverage attestation — review.json gains an optional coverage[] checked against the bundle's own hunks, with --require-coverage to refuse a partial claim and one body line on a partial attestation. Diff coverage
    • ranges: null is malformed — a coverage claim whose ranges is present but not a string now counts as unattested instead of as the whole file. Diff coverage
    • Ignore overrides — repeatable --ignore <glob> and --no-default-excludes on fetch, both recorded in the exclusion receipt. Exclusions
    • Suppression receipts — excluded diff bytes and withheld findings are receipted, with post --min-severity and report --include-withheld to read them. Exclusions
    • Proposed learnings — propose inactive one-rule packs from resolved threads, accept them through the pack importer, or reject them permanently with a reason. Proposed learnings
    • Rule import — learn --pack <dir> --id <rule> imports a structured rule object and keeps its pack provenance and hash. Import a rule object
    • Rule exposure — learn --status gains recalled_in, cited_in, exposure, and acceptance per rule, reporting unmeasured recall as null rather than zero. Knowledge base
    • Risk scoring — an optional risk claim in review.json plus report --risk and report --max-risk, which reports the larger of your claim and a deterministic floor. Triage risk queue
    • Reaction proposals — opt-in reaction proposals from posted comments, plus per-rule acceptance and false-hit rates. Reaction proposals
    • Resolution metrics — live resolution rollups, clean-review metrics, and calibration receipts, with clean reports getting their own headline. Resolution and clean reviews
    • Withheld capabilities — post.json records, per capability, what a host could not carry and why, so a degraded publish stays auditable. Guarded submission
    • Payload receipts — a post payload receipt now hashes the rendered review content and the reviewed head, not diff positions or range metadata. Guarded submission
    • Partial publication — post only the comments and summary parts missing from the fetched remote content, and rearm at a new head with rearmed and previous_head recorded. Guarded submission
    • Live authority — post refreshes live reviews and comments first and refuses without writing when that read fails, so a retry no longer needs a re-fetch. Guarded submission
    • Author fingerprint — fetch writes an advisory author.json ({agent, model, confidence, evidence[]}) from five harness signals, suggests a reviewer from another model family, and record.json carries the author/reviewer pair that trend tallies. Bundle namespace
    • Revert memory — a changed path that matches the tracked reverts ledger gives fetch a bounded, fenced reverts.txt, and the review prompt treats a revert as evidence the change broke something once. Reverts
    • Validation pass — a trusted prompts/validate.md pass writes validation.json per sig, rigor_policy floors each severity tier, and post --require-rigor narrows it per invocation: advisory by default, a receipted withhold in gate mode. Rigor floors
    • Published key names — prompts/review.md tags the evidence rung as rigor and the in-bounds flag as in-bounds, and an archtest holds every published key to review.schema.json. Rigor floors
    • Verdict reconciliation — binding verdicts reconcile separately from summary and inline content, so a retried approval does not duplicate a GitLab summary note. Submit per host
    • Plain-body Basis line — record leaves a review.json body with no leading quote block byte-identical and reports no basis-replaced warning, because there is no confidence lead to insert under. Review-input problems
    • Array-valued coverage ranges — next --coverage refuses a coverage[].ranges array on plain and chunked bundles with the same bad-review-input problem record reports, instead of reading it as a silent 0% attestation. Diff coverage
    • Passing gate next step — a passing report --check <tier> names merge — gate passes in the envelope's next and human footer, leaving address open threads for a failing gate. Machine surface
    • More generated paths — *.gen.* and zz_generated.* now count as generated in the shared role heuristic, so next --coverage and the risk floor exempt them like the other generated patterns. Diff coverage
  • Agents and MCP

    • Bounded repair loop — run --rounds N re-prompts an agent with the shape checker's fix list, stops on the first valid round, and refuses repair-exhausted. Your agent, launched
    • Self-review loop — a bounded local self-review recipe for Claude, Codex, and OpenCode with round receipts and coverage bound to the verified diff identity. Agent self-review loop
    • MCP config for six more hosts — mcp config writes snippets for six more hosts and carries --read-only or --operator into the argv it prints. Host config
    • Declared mutation classes — every MCP tool states its local mutation class, and a call hidden by --read-only returns the typed read-only-tool refusal. MCP server
    • Agent preflight — a missing agent binary is caught before prompt assembly, with install guidance, and printed or recorded pass-through arguments are limited to safe model and cost options. Agents, arguments, and --print
  • Hosts and CI

    • Forgejo publish — post submits one atomic Forgejo review with the body, verdict, reviewed commit, and every inline finding, and withholds what the host cannot anchor rather than guessing. Submit per host
    • GitLab publish — each finding becomes one positioned discussion, the review body a summary note, and an approval verdict a real MR approval. Submit per host
    • GitLab incremental — fetch --provider gitlab --base-sha writes the same incremental.diff contract the GitHub compare path writes, and hosts without a compare adapter say so in a note. Providers
    • GitLab readiness and calibration — ready advises on merge requests and plain calibrate joins live GitLab threads through glab, on the same blocker and outcome vocabulary as GitHub. Calibration
    • doctor --provider — preflight one host at a time, with a per-host probe table and credentials reported by presence, never by value. Doctor preflight
    • GitHub Action — the review and merge gate ships as a first-party composite action that installs a pinned verified release and uses your runner, your agent, and your keys. The runner is your CI
  • Serve and the browser UI

    • verdict serve — a read-only HTTP surface over the current repository, loopback by default, with a session-cookie exchange, strict CSP, Host and Origin gates, and per-response source traceability. Serve
    • Root discovery — serve --root finds nested repositories under a bounded traversal backed by a local 24-hour registry, and reports cache, truncation, and skipped patterns. Serve
    • Cost and precision buckets — /v1/cost adds repository, ISO-week, and model buckets and /v1/precision adds category and tier counts, keeping unmeasured prices null and marking corrupt aggregates partial. Serve
    • Browser UI — serve opens an embedded UI with inbox, review, threads, cost, precision, and gate views, dark mode, live refresh, and static data.json reading. Serve
    • Live invalidation — bundles invalidate as they change, with --interval, --sweep, and --no-watch to control it. Serve
    • Vanished thread feeds — a thread feed removed mid-read now reads as absent instead of corrupt. Serve
    • Cache content budget — serve cache hashing is bounded to 16 MiB per bundle lookup and bypasses the cache when the budget is exceeded. Serve
    • Small-file rewrites — artifacts up to 64 KiB are hashed, so a rewrite that preserves size and mtime is still detected. Serve
    • Inventory accounting — serve inventory accounting stays inside its byte budget even for oversized artifacts. Serve
  • Lens packs

    • Built-in lenses as data — perf, api-contract, and test now ship as a pack resolved into every bundle as lenses.txt, with the four hashed prompt files byte-identical. Lens packs
    • No per-lens cap — a lens returns every finding its evidence supports: the built-in pack declares only max_files: 12, and the dispatch template leads with the lens's domain. Lens packs
    • pack add and a lock — vendor a pack from a directory or git source, validated before it loads and pinned in a strict .verdict/verdict.lock. Lens packs
    • Layered packs — pack list reports every loaded pack with its layer, source, tree hash, ids, and drift, and a repo pack is read from the trusted base so a PR cannot edit the pack that reviews it. Lens packs
    • pack test — run a pack's eval cases offline. Lens packs
    • Leftover backup sibling — a failed sweep of a previous tree is reported as success plus a warning naming the leftover, instead of refusing an add whose new tree is already durable. Lens packs
    • Stale staging — a crashed re-add's leftover staging or backup sibling refuses pack-staging-stale with the path to delete. Lens packs
  • Measurement

    • Clean controls — the benchmark corpus accepts clean controls with empty ground truth, ships two seeded clean cases, and keeps seeded fixtures labeled as not a model-quality claim. Benchmark corpus
    • Reproducible metrics — typed severity weights, ordered k-of-n recall, control false-positive rates, weighted F1, signal-to-noise, and explicit denominators, with provenance and argv recorded. Benchmark corpus
    • Adjudicated rates — a versioned per-run adjudication record reclassifies an unmatched finding as real-unlisted, matches-ground-truth, or false-positive, and bench/runner -render recomputes strict and adjudicated precision and recall without re-running a model. Benchmark corpus
    • First real cases — four real cases from urfave/cli (three bugs and one clean control), -max-usd stops scheduling at measured spend and marks the run uncitable, and refused samples are counted in the cost rollup. Benchmark corpus
    • Corpus grading guard — a keyword the case's own diff spells refuses, a case left with no phrase asks for another keyword, and a real bundle whose full.diff is missing, unreadable, or prose is refused instead of graded leniently. Benchmark corpus
  • Breaking changes (pre-1.0)

    Package is on 0.x, so these ship as a minor bump; each one names the step that migrates it.

    • index blast-radius truncation shape — the integer truncated count and depth_truncated are replaced by the shared boolean receipt (truncated, truncation_reason, returned, total, next_offset), and a complete result omits the receipt instead of carrying truncated: false. Migrate: read truncation_reason and returned; treat an absent receipt as complete. Code index
    • gate checks coverage by default — the new coverage dimension fails when the gated HEAD has no receipt, a receipt for another HEAD, no coverage[] attestation, or under 100% attested. Migrate: --coverage warn, or the SDK Coverage field, keeps the old passing behavior while reviews attest nothing. Gate
    • gate --live reads every host — a non-GitHub bundle used to pass the live dimension without reading threads; it now reads the bundle's own host and fails closed when it cannot. Migrate: --live auto warns and gates on the local dimensions only. Gate
    • Unknown gate tiers refuse — an unrecognized tier is bad-flags on the MCP report tool and in Report.Check/Gate.MaxSeverity, not only on the CLI. Migrate: send a tier from the published set, or omit the field. Gate
    • verify child environment is default-deny — only baseline locale, terminal, shell, XDG, and toolchain names pass, and a secret-shaped name is dropped even when it matches an allowlist prefix. Migrate: list what you need in .verdict/verify-env or --env-allowlist; dropped names appear in warnings[]. Child environment
    • verify refuses command shapes it used to run — wget --remote-name/-i, curl -K, busybox wget, pip3 download <url>, telnet, git archive --remote, and the rest refuse verify-networked in attached and clustered spellings, and a command word built from a variable (E=rm; $E -rf ./target, including the braced, quoted, exported, loop, and wrapper spellings) refuses verify-indirection. Migrate: write the tool word plainly and rewrite the command to a plain fetch your recipe can run; every refusal keeps its try: hint. Known residuals
    • .verify.out is owner-only — the verify stream is written 0600 on Unix where it was 0644, because a command that prints a credential leaves the value in it. Migrate: read it as the user that ran verify, or copy the bytes inside that same step; on Windows the file keeps the directory's inherited ACL. Enforced sandbox
    • Bundle paths are guarded — a bundle name that is not one path element (a/b, .., CON, x:) refuses bad-flags, and a symlinked bundle root or component refuses before any write, including a link that points inside the repository. Migrate: one plain name per bundle, and a real directory where a link used to work. Bundle namespace
    • Provider is inferred from origin — with no --provider flag, an unknown or conflicting origin now refuses provider-unknown instead of defaulting to GitHub. Migrate: pass --provider explicitly, or keep a plain GitHub origin. Providers
    • capabilities splits, truncated widens — the binding verdict becomes approve and request_changes with review_verdicts kept as the compatibility sum, unsupported names each absent capability, and data.truncated widens from boolean to boolean | row[]. Migrate: branch on the two new flags, and narrow truncated before you read it as a boolean. Published schemas
    • pack add source allowlist — only https://host/path with no userinfo, ssh://user@host/path, and user@host:path are accepted; anything else refuses before a git call, and a lock row carrying one refuses at pack list, fetch, post, and record. Migrate: rewrite the source in one of the three forms. Lens packs
    • Benchmark cases name their commits — a real corpus case now requires provenance.base_sha and provenance.head_sha (40-hex), the two commits the reviewed diff spans, and a bundle stamps the same pair. Migrate: add both commits to every real case.json; a case without them fails validation. Benchmark corpus
    • Rebuild indexes — indexes written by the previous scanner over-report callers, so rebuild them. Migrate: verdict index build --at HEAD. Code index

Patch Changes

  • Safety and trust

    • Tokens off the argv — Forgejo and Bitbucket reads hand the credential to curl on stdin or a 0600 file, so it is gone from ps and /proc/<pid>/cmdline. Providers
    • Secret guard — post refuses a secret-shaped value before the first host call, naming the shape and never the value, with --allow-secret-values as the explicit exception. Guarded submission
    • Mentions neutralized — record escapes @-mentions once, where the body is built, so recorded, posted, dry-run, and digested bytes cannot diverge. Mentions
    • Punctuated mentions — (@alice), **@alice**, and _@alice_ are escaped while real addresses stay byte-identical. Mentions
    • Strikethrough mentions — ~~@alice~~ is escaped too, while a lone tilde in prose stays as written. Mentions
    • Learning evidence — cited paths and comment identifiers must stay inside quoted learning evidence, and unsafe ones refuse before a learning is proposed or accepted. Learnings
    • Unicode line breaks — U+2028 and U+2029 now refuse a serve URL, refuse a learning path, and flatten in a rendered pack test line. Serve
    • Store locking — every .verdict/kb.jsonl write takes one advisory lock; a second writer refuses store-busy instead of dropping a row. Findings store
    • Enforced sandbox — verify --sandbox runs the verify command under bubblewrap on Linux with no network, a private PID namespace, empty ~/.ssh/~/.aws/~/.docker, and no silent fallback. Enforced sandbox
    • macOS sandbox — verify --sandbox now denies every bind() and connect() on macOS through the built-in sandbox-exec, loopback and unix sockets included, so filesystem isolation stays Linux-only. Enforced sandbox
    • Variable command words refuse — E=rm; $E -rf ./target and the braced, quoted, exported, loop, concatenated, and own-line spellings now refuse verify-indirection, including after env/timeout/xargs wrappers and inside sh -c. Known residuals
    • .verify.out is owner-only — the verify stream is written 0600 on Unix, like .run.out, because a command that prints a credential leaves the value in it. Enforced sandbox
    • Launch-record redaction — a secret-shaped run --agent operand no longer reaches the recorded output_tail or the 0644 child log. What run asserts
    • setup-verdict hardening — the action passes its inputs through environment variables and verifies a cached archive before reuse, failing closed on a tampered one. The runner is your CI
    • Install preflight — skill install probes exactly the directories it will write before the first byte, so a read-only target leaves nothing half-installed. Install routes
  • Fixes

    • Read-verb refusals — report, next, and show refusals carry a shell-quoted re-enter: line naming the invocation you typed. Machine surface
    • next --coverage with no positionals — leads with one error: line naming what is missing, and a plain Basis: line is accepted without a warning. Machine surface
    • Unnamed cost models — record warns once per cost.json models[] entry with no name, and still accepts and prices the record. Cost control
    • Over-cap and missing-index refusals — diff-too-large and missing-index now carry a next: line naming the exclusions that fit, and the exact build command. Over-cap diffs
    • Five small repairs — an agent that fails just before its budget reads agent-failed, MCP refusals carry their warnings, index-capped names the bytes it measured, index status points at the query, and a crashed agent's partial review can no longer block on a FIFO. Machine surface
    • Windows watchdog — a command that failed to start and survived the cleanup kill now returns at once, naming both errors, instead of hanging forever. Bounded foreground waits
    • Quoted upgrade hints — verdict upgrade quotes its project-local cd hint so a path with a space or quote pastes resolvably, and the uninstall sibling hint reads as a sentence. Machine surface
    • Release asset check — the release script's completeness check no longer reports a shipped asset as missing when the asset list is large. Release train
    • pr-wait inconsistent reads — a head that is not yet the same on both APIs is retried instead of failing the bounded wait, and inconsistent_observations keeps the retry visible. Bounded foreground waits
    • Windows watchdog resume — a command that failed before starting under load retries the thread lookup instead of refusing resume suspended verify process. Bounded foreground waits
    • backtest mine in a partial clone — a --filter=blob:none clone refuses partial-clone with the step that fetches the missing objects, instead of stalling. Backtest
    • Quiet local live skip — a local bundle's live recheck writes one info: line to the human stream instead of a warning or a --json warnings[] entry; failed remote reads still warn. Machine surface
    • Optional review schema version — record accepts a review.json without schema_version and reads it as version 1, while a present value other than "1" still refuses. Review-input problems
  • Docs

    • Refusal code reference — docs/troubleshooting.md lists every refusal code with its cause and fix, and a drift test fails the build when one is missing. Refusal codes
    • Docs site structure — grouped sections with short sidebar labels, a Markdown twin for every page, and a glossary route from verdict --help. docs/
    • Defaults at a glance — a generated table names every default a bare run takes and the verb it shapes, so it cannot drift from the shipped binary. Defaults
  • Known limits

    • The macOS sandbox covers the network half only — verify --sandbox denies bind() and connect() on macOS but has no filesystem namespace, so credential paths, /proc, /run, and a private /tmp are hidden only on Linux under bubblewrap; filesystem isolation on macOS needs a Linux VM or a container running that recipe. Enforced sandbox
    • The string gate is a tripwire, not a boundary — sh "$V", a script named by a variable, still passes the pre-exec scan, which is why verify --sandbox is the boundary and a spelling outside the gate's reach is a gate gap to arm rather than a vulnerability. Known residuals
    • A backslash before a CRLF line ending is not joined as a continuation — the gate refuses that shape rather than passing it, so a CRLF continuation fails safe. Known residuals