Skip to content

feat(baseline): record the findings that already exist, so only new ones fail - #460

Merged
wenzowski merged 4 commits into
mainfrom
wenzowski/cloud-67-add-a-baseline-command-for-already-dirty-repos
Aug 14, 2026
Merged

feat(baseline): record the findings that already exist, so only new ones fail#460
wenzowski merged 4 commits into
mainfrom
wenzowski/cloud-67-add-a-baseline-command-for-already-dirty-repos

Conversation

@wenzowski

Copy link
Copy Markdown
Contributor

Closes CLOUD-67.

Adopting Batten in an existing repository had two routes to green: fix every
pre-existing violation at once, or weaken policy. This adds the third — a
baseline, the persisted set of finding identities that already existed, so
check stops failing on them and still fails on anything new.

What keeps this inside the threat model

A baseline is a bulk waiver by another name, and the agent running it is both the
finding's subject and the store's writer — the issue's own adversarial review
ranks that the top risk. So the load-bearing part is not the filter, it is the
minting predicate: only landed, committed state may be baselined.

The Ready block asks for "an ancestor of the authoritative ref". That cannot be
written as stated — git.rs's no_ancestry_decides_merged_ness forbids a
reachability verdict crate-wide, and a rebased or squashed landing is invisible
to ancestry on a fast-forward trunk anyway. The predicate is therefore spelled as
patch identity, reusing worktree::status (git::landing plus the
uncommitted and unpushed facts). Any at-risk work refuses the mint, exit 2,
writing nothing. Unlanded::NotComputable refuses too: unproven is not clean.

The rest of the shape

  • Filter placement. In run_rules, immediately before the waiver filter.
    The order is load-bearing in one direction: a waiver removes a finding, so
    waivers first would make a live baseline entry read as unmatched.
  • Counts reuse CLOUD-123's direction-aware semantics wholesale — an increase
    re-raises (new evidence fails), a decrease ratchets and surfaces only as prune
    staleness, zero resolves.
  • Staleness is an ordinary Finding (Scope, baseline.stale), joined
    where budget and defects join, so it inherits waivers, -J, the exit
    contract and the store rather than re-implementing them.
  • Two fail-closed holds, neither ever pruned: a rule in not_evaluated holds
    its entries (silence is not evidence), and an entry minted under a superseded
    identity_version holds as baseline.version-drift instead of silently
    unmatching — the issue's "a bump must not invalidate every adopter's baseline".
  • Count drift never moves a tierseverity.rs's deferred invariant, now
    landed. Structural: apply only removes elements and never builds or mutates a
    Finding.
  • Storage is one JSON document under the bound store, beside findings/
    and journal/, rather than a plain repo_state_dir join like every other
    store here: it keys on finding identities, so a baseline surviving a
    state adopt would describe a store the checkout no longer owns.
  • Output is pointer-onlyrule <digest12>, never a baselined line.

Two things the gates caught, worth reading

  • no_ancestry_decides_merged_ness failed on the module header, which
    spelled the forbidden vocabulary out in order to explain it. The gate was
    right; one that exempted the prose describing it would have a hole shaped
    exactly like a comment.
  • perf-gate measured the first draft putting store::resolve on every check:
    p50 3.25ms → 24.93ms, a 7.7x regression on the workhorse verb. The guard added
    for it is exact rather than a heuristic — every directory bound_dir can name
    is a store directory under the state root, so if none carries a baseline.json
    there is nothing to load. One read_dir, no process spawn.

Verification

mise run verify green (the earlier sonar-gate refusal was "not pushed yet").
813 lib tests, every integration suite, 1815 bats cases, perf-gate back within
threshold. The e2e suite is crates/batten/tests/baseline.rs — kept out of
tests/cli.rs per waivers.rs's stated precedent — and each refusal case
asserts the absence of the artifact as well as the exit code, because
asserting the code alone would pass for an implementation that refuses and writes
anyway.

House style §2's command tree gained the baseline row in the same change, which
is what the_emitted_surface_is_exactly_the_committed_row_set exists to prompt.

@linear-code

linear-code Bot commented Aug 14, 2026

Copy link
Copy Markdown
CLOUD-67 Add a `baseline` command for already-dirty repos

Why
Adopting Batten should not require a single big-bang cleanup. Existing violations need a baseline path so only new ones fail.

Acceptance

  • Existing violations can be baselined
  • New violations still fail after baselining

Refinement — Ready

Refinement gate: Definition of Ready & Done. This body carries only specializations.

  • Source of truth (§1). The usage spec (the new baseline verb and its --prune flag) and the findings store in crates/batten/src/state.rs. The baseline file itself is a derived store, not a hand-authored artifact.
  • Computable predicate (§2). batten check exits 0 when every finding's identity is present in the baseline and non-zero on any identity that is not — an exit-code predicate over the persisted identity set, never a judgement about whether a violation "matters".
  • Effect (§3). baseline self-declares write: it persists to the out-of-tree per-repo state directory. baseline --prune stays write, not destructive — it removes only entries whose backing finding no longer exists. check is unchanged and stays read.
  • Generated artifacts (§4). A new verb regenerates completions, man pages and the config schema, each diffed byte-for-byte against the committed copy.
  • Output & exit (§5). Baseline findings are pointer-only — rule id and identity digest, never the matched content. An unmatched baseline entry is a finding on the 0/1/2/3 table, not a warning line.
  • Commit / bump (§6). feat → patch until 0.1.0.
  • Test obligation (§7). End-to-end over the compiled binary: baseline a dirty tree and assert check exits 0; introduce one new violation and assert it still fails; resolve a baselined violation and assert the unmatched entry is reported; assert --prune clears it. The minting predicate gets its own case: a baseline write against a state that is not an ancestor of the fetched authoritative ref must be refused.
  • Blockers (§8). blockedBy CLOUD-123 — the identity function the baseline keys on. Implementation is additionally gated on CLOUD-169's fixture pack being green, which is a relatedTo sequencing constraint rather than a blocking dependency.

Inherited constraints from the identity decision (CLOUD-123) — a baseline is a persisted set of finding identities, and its governance is part of the threat model:

  • The baseline keys on the decided identity function (CLOUD-123); "new violations still fail" is exactly the same-or-different decision. An identity-version migration must explicitly cover baselines or a bump silently invalidates every adopter's baseline.
  • Staleness is loud, never silent: unmatched baseline entries surface as errors with a --prune verb (the convergent behavior of PHPStan reportUnmatchedIgnoredErrors, Psalm --update-baseline, ESLint/stylelint --prune-suppressions).
  • Count-tier degradation (Psalm model): when per-occurrence content data is inconsistent, degrade to count-based suppression per (file, rule) rather than failing; count drift may invalidate a baseline entry but never moves a severity tier.
  • Minting governance (adversarial-review finding, top-ranked risk): the agent is both the finding's subject and the store's writer — the most likely honest error is greening a red gate via batten baseline instead of fixing the defect. Baseline writes are gated by a computable local predicate: the baselined state must be an ancestor of the authoritative ref at its currently-fetched SHA, with (ref, sha, commit, fetch_time) recorded in the journal; fetch staleness surfaces loudly. PR-review state is advisory only (server state is not a local predicate); signed tags/trailers if offline review evidence is required.
  • Blocked-by CLOUD-123; no baseline mints until CLOUD-169's fixture pack is green.

Review in Linear

@wenzowski
wenzowski marked this pull request as ready for review August 14, 2026 20:47
@wenzowski
wenzowski force-pushed the wenzowski/cloud-67-add-a-baseline-command-for-already-dirty-repos branch from 990978a to c625ffd Compare August 14, 2026 20:47
…nes fail

A repository adopting Batten had two routes to green: fix every existing
violation at once, or weaken policy. This is the third — a baseline, the
persisted set of finding identities that already existed, so `check` stops
failing on them and still fails on anything new.

A baseline is a bulk waiver by another name, and the agent running it is both
the finding's subject and the store's writer, so the load-bearing part is not
the filter but the minting predicate: only landed, committed state may be
baselined. It is spelled `worktree::status` — patch identity through
`git::landing` — because `no_ancestry_decides_merged_ness` forbids a
reachability verdict crate-wide, and a rebased landing is invisible to ancestry
anyway. An unresolvable target refuses too: unproven is not clean.

Drift reuses CLOUD-123's direction-aware counts wholesale — an increase
re-raises, a decrease ratchets and surfaces only as prune staleness, zero
resolves — and staleness is an ordinary Finding joined where budget and defects
join, so it inherits waivers, -J, the exit contract and the store. The filter
sits immediately before the waiver filter, and that order is load-bearing:
waivers first would make a live entry read as unmatched.

Two fail-closed holds, neither ever pruned: a rule in not_evaluated holds its
entries, and an entry minted under a superseded identity_version holds rather
than silently unmatching.

Refs: CLOUD-67
`run_rules` is the one funnel `check` and `enforce` share, and it was already
at the line bound; the filter reads better as its own function anyway, where the
one thing a reader must not get wrong — that it runs BEFORE waivers, because a
waiver removes a finding and would make a live entry read as unmatched — is
stated at the definition rather than buried mid-funnel.

Refs: CLOUD-67
… forbids

`no_ancestry_decides_merged_ness` scans src/ including comments, and the
paragraph explaining why reachability is forbidden spelled the vocabulary out to
explain it. The gate was right: one that exempted the prose describing it would
have a hole shaped exactly like a comment.

Refs: CLOUD-67
perf-gate measured what the first draft cost: putting `store::resolve` on every
`check` took p50 from 3.25ms to 24.93ms, a 7.7x regression on the workhorse
verb — the resolution spawns git three times and may walk every store, to answer
a question that is almost always "no baseline".

The guard is exact rather than a heuristic: every directory `bound_dir` can name
is a store directory under the state root, so if none of them carries a
baseline.json there is nothing this checkout could load. One read_dir, no
process spawn, and a false is a proof.

It guards `load` rather than `path` because the two callers want opposite things
from an empty machine: `check` wants to spend nothing finding out there is no
baseline, and `save` is on its way to writing the first one.

Refs: CLOUD-67
@wenzowski
wenzowski force-pushed the wenzowski/cloud-67-add-a-baseline-command-for-already-dirty-repos branch from c625ffd to 91e3d9a Compare August 14, 2026 21:16
@sonarqubecloud

Copy link
Copy Markdown

@wenzowski

Copy link
Copy Markdown
Contributor Author

/fast-forward

@wenzowski
wenzowski merged commit 91e3d9a into main Aug 14, 2026
9 checks passed
@wenzowski
wenzowski deleted the wenzowski/cloud-67-add-a-baseline-command-for-already-dirty-repos branch August 14, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant