Skip to content

v3.7.0

Choose a tag to compare

@its-miso its-miso released this 20 Oct 17:23
· 398 commits to main since this release

3.7.0 is about the verdict being real. Until now, review_verdict could reach approve and still publish a non-blocking comment, the only judge of a finished review was the model that wrote it, and what happened to a finding after publication was invisible. This release publishes the verdict it reached (plus a new blockers-only mode), adds an opt-in independent scorer that grades completed reviews, and exports a finding-outcome contract so operators can measure what actually happens to findings. Around that: a credential-redaction wave closes every remaining path where someone else's secret could ride to disk, and the self-hosted controller gains repository acquisition, GitHub App auth, and its first Tangled publish record.

If you publish verdicts or want the scorer, this is the release to pin:

- uses: misospace/pr-reviewer-action@v3
  with:
    publish-mode: review_verdict   # an 'approve' verdict now publishes a real approval
- uses: misospace/pr-reviewer-action@v3
  with:
    ai-confidence-model: ${{ vars.CONFIDENCE_MODEL }}       # empty (default) = scorer off
    ai-confidence-base-url: ${{ vars.CONFIDENCE_BASE_URL }}  # a distinct backend grades independently
    ai-confidence-api-key: ${{ secrets.CONFIDENCE_API_KEY }} # a distinct URL never inherits the primary key

The verdict publishes as the verdict

  • review_verdict is authoritative (#1162, closes #1161): a valid final verdict maps to its native event — approve → APPROVE, request_changes → REQUEST_CHANGES — instead of quietly downgrading to a comment. The one remaining comment path is deliberate scope policy: a fork PR without approve-forks, or unknown fork-ness, still fails closed to an advisory comment. A missing or invalid verdict publishes neither event and says so. allow-approve is deprecated and inert; setting it logs a deprecation warning.
  • review_blockers publishes blocking requests only (#1162): a request_changes verdict becomes a native blocking request, an approve verdict becomes an advisory comment, and the mode never publishes an approval under any verdict, input, or PR scope.
  • Shortfalls are named, not silent (#1164, closes #1145): an approval published with incomplete coverage now carries an explicit "Approval published with incomplete coverage" note instead of being downgraded to a comment that looked clean; a run with no valid verdict fails with a machine-readable error rather than posing as a successful comment review.
  • Evidence gaps get one bounded repair round: after the verdict turn accepts a response, a deterministic probe can re-run requirement-trace validation over the accepted coverage and, on a genuine gap with budget remaining, append an untrusted-fenced instruction to the same conversation for one bounded tool-loop round before the verdict is re-requested. Every outcome — continued or skipped, and why — is recorded and sanitized on the artifacts. And a requirement truth re-derived from the checkout and found genuinely absent (structural-proof-unconfirmed) now synthesizes a concrete blocker and flips approve to request_changes, while a reviewer's own citation-scope note (#854) no longer does — it stays a repairable coverage gap the continuation can go investigate.

A second opinion, opt-in

  • Independent post-review confidence scorer (#1140, closes #1073): after the review finishes and before publication, a separately configured model receives a credential-redacted, untrusted-fenced summary of the final review and returns a 0–5 score, a risk band, a bounded reason, and bounded concerns. ai-confidence-* inputs configure the lane; a distinct ai-confidence-base-url never implicitly inherits the primary credential. The result is advisory telemetry only — it never touches the verdict, findings, or any gate — and it is off by default: unset means no extra model call and empty outputs, never a spurious 0. On fork PRs the scorer is off unless explicitly enabled, because its prompt sends the review's own privileged output to another endpoint.

What happens to a finding afterwards

  • Finding-outcome and feedback data contract (#1115, closes #1027): every published inline finding carries a stable, action-owned identity marker, and a successful publish now exports finding-outcomes.json — a finding-outcomes/v1 event ledger (reported, repeated, resolved, dismissed, feedback, …) with deterministic reconciliation. Privacy-aware by construction: events carry digests and counts, never message text or logins, and adjudicated precision reads only explicit operator adjudications — a resolved thread or a 👎 cannot inflate it. No verdict, severity, suppression, or prompt behavior changes; this is the measurement seam the v3.6.0 feedback loop was missing.

Credentials that aren't yours still get masked

  • PR-authored content is redacted at ingestion (#1157, closes #1154): the diff, PR body, and every artifact derived from them now pass a credential-shape redactor once, at ingestion — so the model input and every persisted artifact (pr.diff, ai-request.<tier>.json, corpus sections) carry the same [REDACTED] bytes and cannot diverge. Over-redaction of a credential-shaped test fixture is the accepted tradeoff; the configured-key layer stays out of the diff so a short key can't destroy review input.
  • Provider-authored raw bodies get the generic pass (#1153, closes #1149): ai-response.<tier>.json on every lane now goes through redactTextDeep, so a credential the provider invented — a key it generated, another tenant's token quoted in an error body — can no longer persist verbatim even though this run never configured it.
  • Linked-issue bodies redact at the projection seam (#1174, closes #1159): fetched linked-issue titles, labels, and bodies reached the corpus with no redaction at all; the single projection feeding every sink now masks same-line and split-line credential shapes with line counts preserved, before any embed cut.

Model-authored markdown can't forge a prescription

  • Reservation follows the container, not just the line (#1135, closes #1112): a ```suggestion opener behind a blockquote or list marker rendered as a live one-click prescription cmark-gfm honors but publication never validated. The opener is now matched behind container prefixes, with the prefix preserved so benign prose stays byte-identical.
  • An unclosed fence can't swallow the validated prescription (#1139, closes #1136): a model message ending in an ordinary fence with no closer extended to end-of-document and rendered the action's appended, validated prescription as inert fence content. Unclosed ordinary fences are now dissolved like reserved ones — deterministic, and benign closed blocks stay untouched.
  • Merge and prescription keys are structurally injective (#1150, fixes #1119): both key builders joined model-authored fields with a bare NUL, and a NUL is not JS whitespace — a model could shift a field boundary and fold a distinct finding into a "duplicate". Keys now use array serialization, matching the outcomes-identity precedent.

Fail loud, not silent

  • Platform response bodies are bounded while streaming (#1169, closes #1131): a 32 MiB cap is enforced on bytes as they arrive, independent of Content-Length. The part that matters: an oversized diff previously degraded to an empty string, fingerprinted as empty-diff, and would have silently skipped every later push on the PR — the adapters now rethrow and the run fails visibly.
  • The durable queue refuses bad snapshots (#1171, closes #1130): persisted job records are validated field by field, duplicate generation ids are rejected at intake and adoption, and malformed snapshots fail closed instead of loading.
  • Supersession-aborted generations retire (#1168, fixes #1141): a superseded executor that reported failure used to requeue as runnable work and could win the next dispatch slot over its own replacement.

Toward the self-hosted controller

More groundwork libraries — exercised by tests, not yet imported by the Action runtime, so Action users should notice exactly nothing here.

  • Repository acquisition (#1125, closes #732): a persistent bare mirror cache with bounded incremental fetch, immutable head/base SHAs verified before any workspace exists, and isolated per-review worktrees with crash-recovery sweeps. Credentials stay per-invocation env; nothing is written to repo config or remotes.
  • GitHub App auth and the capability seam (#1120, closes #1133, substrate of #729): RS256 JWT minting and installation-token lifecycle in-process — no gh, no shell-out — with credentials scoped exactly to the repository each call names, and managed-comment ownership proven per write so a PR author's own marker comment can't be spoofed into an update target.
  • First Tangled publish record (#1118, closes #588): unlike the libraries above, this seam is wired into the publish path — Tangled builds one managed top-level ATProto review record (dev.prreviewer.review) and never routes through the GitHub/Forgejo publish APIs. It stays experimental and fail-closed: with no trustworthy current-head source yet, publication refuses every PR today until #1165 lands. Nothing is expected of Tangled users yet.

Upgrading

Pin to v3.7.0, or the floating v3 tag, which now follows this build. Immutable-version and SHA pins must be bumped deliberately.

This release has real behavior changes. The honest notes:

  • publish-mode: review_verdict users: an approve verdict now publishes a native APPROVE event where it previously published a comment. That is the point of the mode, but if you relied on the old always-comment behavior, switch to the new review_blockers mode — it never approves. allow-approve is deprecated and ignored; fork approvals still require the explicit approve-forks opt-in. Changing publish mode changes the review fingerprint, so the switch triggers one fresh review.
  • Partial-coverage approvals are no longer downgraded: an approved review with coverage gaps publishes APPROVE with an explicit incomplete-coverage note instead of a silent downgrade to COMMENT. If your branch protection treats approvals as merge-eligible, read that note before merging.
  • Artifacts are redacted more aggressively: ai-request.<tier>.json, ai-response.<tier>.json, diff/PR-body artifacts, and linked-issue sections now contain [REDACTED] where credential-shaped values appear — including values you never configured. Anything parsing those artifacts for raw content will see masks. Unset configuration still produces byte-identical request behavior.
  • All new inputs are opt-in with empty/false defaults: the confidence scorer makes no model call when ai-confidence-model is unset, and the confidence-* outputs are empty rather than zero.

Every input's current contract lives in docs/inputs.md.

Full changelog: v3.6.0...v3.7.0

Generated changelog (release-please)

Features

  • #1073: independent post-review confidence scorer (advisory, opt-in) (#1140) (a6a51a2)
  • #588: publish and update the managed top-level Tangled review record (#1118) (3699401)
  • acquisition: repository mirror cache and isolated per-review workspaces (#732) (#1125) (f14a977)
  • ingress: GitHub App auth + capability seam — every forge read/write through the capability, not a CLI (#729) (#1120) (a44ea97)
  • outcomes: finding-outcome and feedback data contract (#1027) (#1115) (bdeab8a)
  • review: enable trusted repository skills (#1134) (0d26862)

Bug Fixes

  • #1112: reserve container-prefixed suggestion openers behind blockquote/list prefixes (#1135) (2d0d39f)
  • #1119: make merge and prescription keys injective against NUL forgery (#1150) (6fade46)
  • #1130: validate persisted job identities and reject duplicate queue ids (#1171) (c8b77a9)
  • #1131: bound platform text/JSON response bodies while streaming (#1169) (1855642)
  • #1136: dissolve an unclosed ordinary fence that would swallow the validated prescription (#1139) (bac85a1)
  • #1141: retire supersession-aborted generations on failed executor settle (#1168) (31342a1), closes #1141
  • #1145: review_verdict publishes the verdict; bounded tool-loop evidence-repair continuation; confirmed-unmet escalation (#1164) (716adb1)
  • #1149: route persisted raw provider bodies through redactTextDeep (#1153) (04e5bd1)
  • #1154: credential-shape redact the PR-authored artifact family (incl. diff path headers) (#1157) (f5f8f74)
  • #1159: redact fetched linked-issue bodies at the projection seam (#1174) (a66b9e9)
  • #1161: publish the review_verdict verdict; add review_blockers mode (#1162) (1ceebd8), closes #1161
  • deps: update dependency astro (7.4.0 → 7.4.1) (#1160) (cf8d3f1)

Chores