Skip to content

v3.4.0

Choose a tag to compare

@its-miso its-miso released this 07 Oct 20:33
· 525 commits to main since this release

3.4.0 is about attribution. A reviewer earns its keep by blocking a PR only for something the PR actually did, and by claiming only what it can prove. Both halves got real work here: a proof layer that can now handle a requirement enforced across several seams instead of demanding one blessed location, and a cluster of verdict fixes that stop a red check from landing on the author's doorstep when the real cause was a degraded model run, a check that merely got cancelled, or a question nobody can answer from the diff.

The headline is distributed enforcement for requirement tracing (#981). Declare a rule's seams in .github/pr-reviewer-owners.yml, and the reviewer must cite every seam — its own enforcement location and its own matching test — before it will call the requirement met. A cross-cutting standard that used to be unverifiable by construction is now provable, and only from a topology the model can't author.

Nothing breaks and nothing needs reconfiguring. One default changed in the direction of blocking less, and two opt-in paths shifted; all three are spelled out in Upgrading so a stricter gate than you expect never surprises you.

If your specialists should run on a different model than your primary, that is finally a first-class input:

- uses: misospace/pr-reviewer-action@v3
  with:
    ai-base-url: ${{ vars.AI_BASE_URL }}
    ai-api-key: ${{ secrets.AI_API_KEY }}
    ai-model: ${{ vars.AI_MODEL }}
    # deep-review specialists can run somewhere else entirely
    ai-specialist-model: ${{ vars.SPECIALIST_MODEL }}
    ai-specialist-api-key: ${{ secrets.SPECIALIST_API_KEY }}

Requirements that span more than one place

  • Distributed enforcement (#981, closes #962): a rule in .github/pr-reviewer-owners.yml may declare groups — named seams, each with its own owners globs and optional tests globs. The requirement is met only when the reviewer cites every declared seam with a distinct enforcement location and a distinct matching test (an injective match, so one citation cannot cover two seams). The topology is read from the base ref: the model cannot declare its own requirement distributed, and a model-supplied symbol is excluded from the predicate. Caps at four seams and fails closed past that; a groups block yielding fewer than two valid seams degrades to a plain ownership rule with a warning.
  • Deterministic state proofs (#990, closes #985): a fourth proof shape, structural_state, for requirements whose truth is a property of repository state rather than runtime behavior or a dedicated test — ".dockerignore must not exclude assets/" is provable by re-deriving the assertion from the cited non-source file. The classifier is deliberately strict: containment is enforced (paths with .. or symlink components fail closed), compound assertions are rejected, and a trailing test-demand adjunct is stripped before the polarity check.
  • A verdict with an empty trace no longer reads as a coverage gap (#963, closes #959): when the reviewer cited real locations in prose but the trace array came back empty, the review now says not-traced-by-reviewer instead of falsely reporting "no valid enforcement location", and a bounded one-pass repair re-asks the primary route for the missing ids. It never overwrites a claim that already exists, and it stands down if you have replaced the trace fragment.
  • Hyphenated code spans are atomic (#1001): `api-key` no longer splits into the prose words api and key, which was enough to let a fabricated "api key" phrase scope a requirement into a redaction-only diff and raise a false requirement_trace_incomplete.

All of this stays behind requirement-trace: true, as before. Nothing here turns on by default.

The reviewer stops blocking on things the PR didn't do

  • Verification asks never block (#986): a category: verification finding is excluded from every blocking determination. Under the default strict policy a finding set of only verification asks publishes as approve; findings_severity_gated never escalates on them; verdict-policy: model relaxes a model request_changes when the entire open set is asks. A major verification finding could block a PR before — it cannot now.
  • A degraded run cannot fail the gate (#987): a request_changes produced by a degraded model run — the availability fallback with no evidence, or on-model-failure: notice — no longer trips fail-on-request-changes. The bypass is provenance-gated on the verdict source, so ordinary primary and fallback verdicts are untouched, and deterministic blocks (evidence blockers, a required check in mode: fail, an incomplete requirement trace) still block.
  • CI restatements stop counting as review findings (#995): check conclusions are carried raw instead of being collapsed to failure, so cancelled, timed_out and action_required read as themselves; superseded runs are deduplicated by name keeping the latest; every run of the reviewer's own workflow is excluded. The section is framed as context-only, and a finding with no code anchor whose message is nothing but a check name plus a status predicate is capped to info. A request_changes whose every finding just re-announces a red check now approves — CI already gates that merge on its own. The match is anchored and strict on purpose: false negatives beat deterministic false approvals.
  • Non-retryable HTTP errors fail fast (#987, closes #978): a 403 exits immediately instead of walking the whole retry ladder — roughly ten minutes at the defaults — before the fallback is even attempted. Retryable statuses (429, 500, 502, 503, 504) are unchanged.
  • A ../ inside a comment is not a path-handling risk (#968, closes #960): comment spans are blanked before the content classifier scans, so a JSDoc mentioning ../events/types.js stops firing path_handling_changes and stops demanding a must_check on a clean diff. Surviving mentions are still recorded as a discounted provenance signal, and CSS hex colours, URL schemes and unspaced floor division are correctly left as code.
  • The diff stops calling itself truncated (#992): the corpus headings are now # PR Diff and # PR Files, and the truncation caveat comes only from the in-band marker a real cut appends. Small PRs stop claiming their diff was truncated in the model's Unknowns section.

Trust, provenance, and secrets

  • Managed-body provenance is authenticated before a skip (#973): precheck now resolves the run's own forge identity and requires a managed comment's or review's author to match before that body may supply a skip fingerprint or a carried verdict. Copying a published fingerprint into a newer comment no longer makes the next run skip the model and carry an approve. Missing or mismatched ownership fails closed — a fresh review, never a silent skip.
  • Managed reads paginate (#971): on long-lived PRs the latest managed review is found, instead of a stale page-one verdict being carried forward.
  • Redaction covers the verified sk- families (#991, #997, #1002): bare and hyphenated OpenAI sk- keys are masked in published prose, and the dashed families are masked in repository source evidence — sk-proj-, sk-ant-api03-, sk-or-v1-, plus the marker- and shape-pinned sk-svcacct-, sk-admin-, sk-None-, sk-ant-oat01- and sk-ant-admin01-. Entropy and marker guards keep ordinary same-prefix paths like docs/sk-proj-deployment-runbook-notes from being redacted. The accepted residual: an oat01 token continuing past its 40-character body is masked whole rather than partially.
  • Forgejo discovers its own CI status (#984): the review job no longer deadlocks waiting on its own still-running status. Auto-discovery needs Forgejo v16.0+ (the release that introduced the run-jobs API), and excludes a pending status only when that API proves a single-job run whose html_url path matches the status target_url path on the runner's origin. Multi-job reviewer workflows and older instances still want an explicit CI_STATUS_CONTEXT; older instances warn immediately and fall back to bounded waiting. No new permissions — the call uses the existing token.

Where you can see it

  • Blocking findings land on the failed check (#988, closes #975): every blocker and major becomes a ::error:: annotation with its file and line on the check run, plus a Severity / Category / Message / File table in the Actions step summary. Model-derived text is redacted and escape-flattened first, so a finding cannot forge a newline or break out of its fence to promote itself into workflow output.
  • Specialists can run on their own model (#979, closes #966): ai-specialist-model, ai-specialist-base-url, ai-specialist-api-format and ai-specialist-api-key define a specialist profile; ai-specialist-correctness-model, ai-specialist-security-model and ai-specialist-tests-model override the model for one role. Unset fields inherit the primary route. Invalid specialist config fails soft back to primary inheritance — it can never redirect the main review. Workflow inputs only: never PR-controlled, never repo-configurable.
  • Drafts are skipped deterministically (#972): no review attempt at all, including a forced ai-review label or /ai-review command. Marking the PR ready re-opens it. If you added your own if: !github.event.pull_request.draft guard, you can drop it.

Routing

  • auto always starts on primary (#965): there is no longer a deterministic pre-primary route to the smart model. Smart is reached only when a completed primary verdict sets smart_review_requested, bounded to one rerun — the reviewer-requested escalation contract from #721, now with no back door. The inert escalate_on_* sibling flags are formally gone from the contract. This only affects you if you set review-routing-mode: auto.

Upgrading

Pin to v3.4.0, or the floating v3 tag, which now follows this build. Immutable-version and SHA pins must be bumped deliberately.

Three behaviors changed, all in the direction of blocking less. None require a config change.

  • verdict-policy: strict (the default) — a category: verification finding no longer blocks, even tagged major (#986). A verification-only finding set publishes as approve-with-findings.
  • fail-on-request-changes: true (opt-in) — a request_changes from a degraded run (availability fallback with no evidence, or on-model-failure: notice) no longer fails the check (#987). Deterministic blocks still do.
  • review-routing-mode: auto (opt-in) — PRs that could previously skip straight to the smart model now always get a primary review first (#965).

Worth knowing:

  • No input renames or removals. Every input's current contract lives in docs/inputs.md.
  • New keys are additive: the ai-specialist-* inputs, and groups inside .github/pr-reviewer-owners.yml.
  • On Forgejo, CI auto-discovery requires v16.0+; older instances warn and fall back to CI_STATUS_CONTEXT.
  • The retired parity/snapshot harness is gone (#983). npm run typecheck && npm test is now the sole developer gate, and the coverage it pinned lives on in focused tests-v3/ regressions.

Full changelog: v3.3.0...v3.4.0

Generated changelog (release-please)

Features

  • precheck: skip draft PRs deterministically (#972) (3634ed5)
  • requirement-trace: support distributed enforcement (#981) (306d92c), closes #962
  • specialists: trusted model-profile overrides (#966) (#979) (d791545)

Bug Fixes

  • ci-evidence: carry raw check conclusions (#995) (97a24ba)
  • classification: ignore ../ inside comments (#960) (#968) (811dfcc)
  • corpus: stop labelling every PR diff as truncated (#992) (605b7ee)
  • forgejo: auto-discover own CI status (#984) (e579697)
  • gate: put the blocking findings on the failed check and in the step summary (#975) (#988) (e5106a1)
  • model: stop retrying non-retryable HTTP errors, unblock degraded verdicts (#987) (0911d88), closes #978
  • precheck: authenticate managed-body provenance before skips (#973) (0f5ef1f)
  • precheck: paginate GitHub managed reads (#971) (aac8a77)
  • precheck: paginate GitHub managed reads (#971) (fa8903c)
  • redact: mask dashed sk- key families in source evidence (#997) (54436f3)
  • redact: mask OpenAI-style sk- keys in published prose (#991) (ad8fa7f), closes #989
  • redact: mask remaining verified sk- families (#1002) (fcdf907)
  • requirement-trace: accept deterministic state proofs (#985) (#990) (ce437eb)
  • requirement-trace: repair a verdict with no trace claims (#959) (#963) (079f485)
  • requirements: treat hyphenated code spans as atomic subject terms (#1001) (9ef0c65)
  • routing: drop direct-smart pre-routing (#965) (200ef91)
  • routing: drop direct-smart pre-routing (#965) (bbea21c)
  • routing: encode primary-only first pass in code (735f664)
  • routing: prebuild smart corpus for escalation (f1070f2)
  • verdict: verification asks never block a PR (#986) (9b488d4)

Chores

  • remove the retired parity/snapshot harness (#983) (cbc8f47)