Skip to content

feat(cli): validate --introduced reports only what your stage caused - #93

Merged
rennehan merged 3 commits into
mainfrom
feat/validate-introduced
Aug 1, 2026
Merged

feat(cli): validate --introduced reports only what your stage caused#93
rennehan merged 3 commits into
mainfrom
feat/validate-introduced

Conversation

@rennehan

@rennehan rennehan commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

validate's verdict covers the resulting branch, not your change. On a branch that already carries findings — the normal state of a large imported graph — validate && commit is unusable however correct the change is, and a caller cannot tell "I broke this" from "it was already like this".

Measured on a real branch: an empty stage returns 99 findings and exit 5.

What it does

$ hydrate validate --introduced          # empty stage, 99 inherited findings
No new coherence findings from your staged change.
Valid: your change adds no coherence errors on branch 'cachetools-keyless-rt'.
exit 0

$ hydrate validate --introduced          # after staging one unwired input
1 coherence finding from your staged change:
  [error] unsatisfied_input  NeedsInput.feed: input port NeedsInput.feed has no incoming edge

2 coherence findings already on branch 'sweep-1', not caused by your change.
Invalid: your change adds 1 coherence error on branch 'sweep-1'.
exit 5

Both directions verified against the live service.

Fail-closed, because the errors aren't symmetric

  • inherited misread as introduced → loud: a spurious non-zero exit, complained about the same day
  • introduced misread as inheritedsilent: it still prints, the verdict reads clean, the gate passes where today it stops

So anything ambiguous is classified introduced.

Identity

(code, raw locator), keyed before path resolution — resolution is presentation, can be partial, and a stale index can fold two ids onto one path.

Counted as a multiset, not a set. A locator legitimately carries more than one finding, so with a set a locator going from one finding to two registers as no change — hiding the new one, in the silent direction. Mutation-verified: switching to set semantics fails 7 tests.

The two calls must describe one branch

If the branch moved between the reads, the answers describe different graphs, and attributing across them would blame your stage for someone else's commit — or hide one of yours behind theirs. It refuses to partition and says so. A conservation check on the buckets does the same for a bug in this code. Mutation-verified: removing the version guard fails 2 tests.

JSON

Three buckets as separate arrays rather than one list with a flag, and valid is redefined to match the exit code, with the server's whole-branch answer under whole_branch_valid. Otherwise the most gateable field in the payload would disagree with the exit status beside it.

Default behaviour is unchanged — this ships behind the flag. Flipping the default is the follow-up, and that one changes what exit 5 means for existing callers.

419 tests. fmt, clippy -D warnings clean.

validate's verdict covers the resulting branch, not your change. On a branch
that already carries findings — the normal state of a large imported graph —
that makes 'validate && commit' unusable however correct the change is, and
gives a caller no way to tell 'I broke this' from 'it was already like this'.
Measured on a real branch: an empty stage returns 99 findings and exit 5.

With the flag, the same empty stage exits 0, and a stage that introduces one
unwired input exits 5 naming only that one, with the 2 inherited findings listed
separately as not caused by the change.

The split comes from two server answers — the branch as it stands, and the
branch with the stage applied. The baseline is prepare() over an empty stage,
which is pure and already a supported request shape, so the real stage is never
touched, swapped or restored.

Fail-closed, because the two error directions are not symmetric. An inherited
finding misread as introduced is loud: a spurious non-zero exit someone
complains about the same day. An introduced finding misread as inherited is
silent: it still prints, the verdict reads clean, and the gate passes where
today it stops. So anything ambiguous is introduced.

Identity is (code, raw locator), keyed BEFORE path resolution — resolution is
presentation, it can be partial, and a stale index can fold two ids onto one
path. Counted as a multiset rather than a set: a locator legitimately carries
more than one finding, so with a set a locator going from one finding to two
would register as no change, hiding the new one in the silent direction.

The two answers must describe the same branch version. If the branch moved
between the reads they describe different graphs, and attributing across them
would blame the stage for someone else's commit or hide one of its own behind
theirs — so it refuses to partition and says why. A conservation check on the
buckets does the same for a bug in this code.

JSON carries the three buckets as separate arrays rather than one list with a
flag, and valid is redefined to match the exit code, with the server's
whole-branch answer kept under its own key — otherwise the most gateable field
in the payload would disagree with the exit status beside it.

Default behaviour is unchanged; this ships behind the flag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rennehan rennehan added the enhancement New feature or request label Aug 1, 2026
@rennehan rennehan self-assigned this Aug 1, 2026
rennehan and others added 2 commits August 1, 2026 15:01
…othing

A review mutated three of these tests and found them decoration.

Dropping 'code' from the finding identity entirely passed all 421 tests. The
test named for it used base=[UNSAT@a], staged=[UNSAT@a, MISMATCH@a], where
multiset counting hands the second entry to introduced whichever key is used.
The discriminating case swaps the code between the reports: with the real key
that is one resolved and one introduced; keyed on the locator alone it collapses
to a single INHERITED finding, which is the silent direction. Now tested, and
the mutation fails it.

The multiset was only half covered: replacing the baseline tally with a plain
insert survived every test, because no test used a baseline carrying two
findings on one key. Now it does.

The conservation checks were tautologies — the loops push each finding exactly
once, so comparing bucket lengths to input lengths is true by construction, and
a review rewrote them to '> total + 9999' with nothing failing. They now compare
independently re-derived tallies, key by key, and are exercised directly with
dropped, invented and mis-attributed buckets. What they deliberately do NOT
catch is a finding moved between introduced and inherited: those are summed
together against the staged report, and attribution is the bucket tests' job.

The version guard printed a warning and fell through to whole-branch semantics,
where the plan called for a retry then a conflict. It now retries once — a
single concurrent commit is the common case — and returns a version conflict
when the branch is still moving, which is what that condition means everywhere
else here and is retryable.

The partitioned JSON carried a 'located' array whose finding_index pointed into
a 'findings' list the payload does not contain, so an authored path could not be
joined to a bucketed finding at all. Each bucket now carries its own path and
path_complete inline.

And validate no longer issues a second request when the stage is empty: an empty
changeset cannot introduce anything, so the baseline is the staged report by
construction. That matters because validate is write-scope gated and rate
limited per principal, and it was the one case where the extra call bought
nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rennehan
rennehan merged commit da2fae6 into main Aug 1, 2026
3 checks passed
@rennehan
rennehan deleted the feat/validate-introduced branch August 1, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant