rivet validate has no 'new diagnostics since <ref>' view — the edit→verify loop must diff 250+ warnings by hand

[avrabe]

Friction (hit repeatedly in the dogfooding loop)

After editing/adding an artifact, the question an agent (or human) actually has is "did my change introduce any new diagnostics?" — not "what are all 257 warnings in the repo?". Today there's no way to ask that:

• rivet validate prints the full set (257 warnings here); you must --format json | jq/grep for your artifact to tell if you added anything. I hit this twice — most recently checking whether a new REQ added an allowed-values warning.
• --explain <ID> is great but needs you to already know the id, and only covers one artifact — it can't answer "did my edit add a warning anywhere (e.g. a backlink/coverage rule firing on a different artifact)?".
• --baseline <name> is a named, cumulative baseline (different concept); --track-convergence is retry-loop detection; rivet impact --since is artifact-change impact, not diagnostics. None give "diagnostics introduced relative to a git ref".

Suggested fix

A diagnostics-diff against a git ref:

rivet validate --new-since <ref>     # show ONLY diagnostics absent at <ref> (default origin/main or HEAD)

Implementation: validate at <ref> (or read a cached baseline), validate now, set-difference the diagnostics by (rule, artifact_id, message). Exit non-zero only if the new set is non-empty (a natural CI gate: "this PR must not add diagnostics"). This is the same diff-vs-base shape as the schema-version-bump gate (#485/PR #487) and would make the edit→verify loop — and "no new warnings" CI gating — a one-liner instead of a jq pipeline.

Found in the hourly dogfooding loop. Related: --explain (#?, single-artifact), --baseline (named baseline).

[avrabe]

Triage pass — premise reproduced and the suggested-fix sketch is well-scoped, but a few points need a maintainer call before this is autonomously pickup-able.

Verified in current main:

• rivet validate --baseline <name> (named, cumulative baseline) and --track-convergence (retry-loop dampening) exist but aren't a diff-vs-git-ref — your premise is correct that neither answers "did my change add diagnostics?"
• rivet impact --since <ref> is artifact-change impact, not diagnostics-set diff — confirmed distinct concept
• So --new-since <git-ref> is a genuinely new view, not a rebranding

Open design choices (the body is concrete on the diff key but silent on these — they're the ones blocking pickup):

1. How to obtain the baseline. Two options at very different blast radii:
   • (a) git worktree add + re-validate at <ref> (correct, slower, needs git+disk)
   • (b) cached on-disk diagnostics snapshot keyed by tree-sha (faster but adds a new on-disk artifact + invalidation rules)
2. Default ref. Body says "default origin/main or HEAD" — those answer different questions (origin/main = "did this PR add"; HEAD = "did my uncommitted edits add"). Pick one default.
3. Diff key. Body proposes (rule, artifact_id, message). The message part will flap whenever a message is reworded — recommend (rule, artifact_id, span) or just (rule, artifact_id) with the new diagnostic's full body shown.
4. Exit semantics. Body proposes "exit non-zero only if new set non-empty." That's the right CI gate, but confirm — --baseline today exits 0 even with new findings (it's informational), so this is a behaviour split.

Could you lift the resolved choices into an explicit ## Acceptance Criteria block? Suggested shape — adjust the bracketed picks:

## Acceptance Criteria
- [ ] `rivet validate --new-since <ref>` (default `[origin/main | HEAD]`) runs validation now, runs validation at `<ref>` via [git worktree | cached snapshot keyed on tree-sha], and prints **only** diagnostics in the set-difference
- [ ] Diff key = `(rule, artifact_id, [span | message])` — duplicates collapsed; output preserves the same text/JSON shape as `rivet validate` so existing tooling keeps working
- [ ] Exit code: 0 iff the new-set is empty; non-zero otherwise (mirroring the schema-version-bump gate from #485/PR #487)
- [ ] Works on a dirty working tree (uncommitted changes) — the "current" side is the working tree, not HEAD
- [ ] Best-effort: if the project isn't a git repo or `<ref>` is unreachable, fall back to "full set" with a one-line stderr note (no hard error)
- [ ] Regression test: fixture project + a stubbed git env with a known prior set; assert the new-set output matches a golden file (text + JSON)
- [ ] `rivet validate --format json --new-since <ref>` emits the new-set as the same `diagnostics: [...]` shape (so `jq` pipelines keep working) and a `baseline: { ref, sha }` field
- [ ] Doc: `docs/validate.md` (or `--help`) explains the three concepts side-by-side: `--baseline` (named cumulative), `--track-convergence` (retry damping), `--new-since` (git-ref diff)
- [ ] Commit trailers: `Implements: REQ-NNN` (file a new REQ for the diagnostics-diff view) + `Refs: #485, #487, REQ-159`
- [ ] `cargo test -p rivet-core -p rivet-cli` green; `cargo fmt --check` / `clippy --all-targets -- -D warnings` clean; `rivet validate` PASS

Process side-rail (same as the other open triage threads this week): https://pulseengine.eu/blog/ is still HTTP 503 (the site also has an expired TLS cert) — same symptom flagged on #479/#422/#420/#408. The binding workflow guidance can't be consulted, so no PR will land from this triage run regardless. Comment is the AC ask only.

Not implemented this run.

---

Generated by Claude Code — issue-triage agent run 2026-06-05.

---

Generated by Claude Code