Skip to content

feat(guard): clippy every workspace, and assert the fuzz target set (#383, #390) - #395

Merged
avrabe merged 1 commit into
mainfrom
feat/v036-scope-assertions
Aug 7, 2026
Merged

feat(guard): clippy every workspace, and assert the fuzz target set (#383, #390)#395
avrabe merged 1 commit into
mainfrom
feat/v036-scope-assertions

Conversation

@avrabe

@avrabe avrabe commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

REQ-GUARD-GATE-EVIDENCE-002 (g) and (h) — obligations 2 and 3 of v0.36.0's bug-fix track (plan #392, first obligation #393).

Both are the same shape: a whole-repo tool whose scope is smaller than its name claims, with the shortfall rendering as the ideal reading.

(g) Clippy linted 1 of 4 workspaces

cargo clippy --workspace --all-targets means "all members of this workspace" — the same scope cargo fmt --all had.

Cargo.toml   codegen-exec-oracle/   codegen-kiln-oracle/   fuzz/

Three workspaces have been unlinted behind a required check reporting green. The fmt half of #383 was fixed and this half left — which is exactly why the requirement says "every whole-repo tool uses discovery, not just the one that was caught".

check_clippy_workspaces.py reuses check_fmt_workspaces.discover and its check(); it does not copy them. Tool wording is injected (tool/verb/fix_hint) with fmt defaults, so the fmt gate is behaviourally byte-identical — verified: its 10-case self-test still passes and it still prints 4 workspace(s) checked, 4 formatted. Copying would put discovery in two files, and two copies that drift apart is how #381 happened.

Measured before wiring — all four workspaces are already clippy-clean under -D warnings (run individually, exit 0 each), so this is not a latent red. Live: 4 workspace(s) checked, 4 lint-clean.

The discriminating self-test case is the runner's exit code: the same tree passes when the runner reports clean and fails when it reports dirty. A wrapper that ignored its runner — how this gate would go vacuous — passes the first and fails the second.

(h) The fuzz target list was never compared to its declaration

fuzz/Cargo.toml declares [[bin]] harnesses; fuzz-nightly.yml runs a hand-written matrix.include. The include form is deliberate and stays (only fuzz_scheduler_solver needs -max_len=128; a flat list would impose that cap on two healthy fuzzers, #361). Its cost is two lists nothing compares.

Drop either side and the nightly stays green having fuzzed a strict subset — no error, no warning, no smaller number in the log, because success is per-leg and nobody counts the legs. Nightly is where this hides longest: fuzz_scheduler_solver was red 97/97 runs from 2026-04-25 before anyone looked, and a silently absent leg is harder to notice than a loudly failing one.

Set equality both directions — declared-but-not-run is a harness never exercised; run-but-not-declared is a leg that cannot build and fails looking like a toolchain problem. Either side empty exits 2: a parse that read nothing is a broken scan, not perfect agreement about nothing — the one way this check could itself go vacuous. 7-case self-test, two file reads, no build.

Two things the gate surfaced by being run

Included rather than reverted:

  • codegen-*/Cargo.lock pinned spar-analysis 0.25.0 against a workspace at 0.35.0 — ten minor versions stale, because nothing routinely built there to refresh them. Otherwise every contributor who runs the new gate gets the same dirty diff.
  • codegen-*/target/ and fuzz/target/ were not gitignored.gitignore:1 is /target, root-anchored. The gap never showed because nothing built there. It does now.

Verified

  • 9/9 guardrail self-tests pass on this tree
  • both new gates run green for real
  • release-plane and human-scoped gates still pass on the edited artifacts
  • workflow + artifact YAML parse

Refs #383, #390

🤖 Generated with Claude Code

…383, #390)

REQ-GUARD-GATE-EVIDENCE-002 obligations (g) and (h), the second and third of
v0.36.0's bug-fix track. Both are the same shape: a whole-repo tool whose scope
is smaller than its name claims, with the shortfall rendering as the ideal
reading.

(g) CLIPPY LINTED ONE OF FOUR WORKSPACES.

`cargo clippy --workspace --all-targets` means "all members of THIS workspace" —
the same scope `cargo fmt --all` had. spar has four:

    Cargo.toml  codegen-exec-oracle/  codegen-kiln-oracle/  fuzz/

so the `Clippy` required check has been linting the root and reporting green for
the repo. Three workspaces unlinted behind a required gate. The fmt half of #383
was fixed and this half was left, which is precisely why the requirement is
worded "every whole-repo tool uses discovery, not just the one that was caught".

`tools/check_clippy_workspaces.py` REUSES `check_fmt_workspaces.discover` and
its `check()`; it does not copy them. The tool-specific wording is injected
(`tool`/`verb`/`fix_hint`) with fmt defaults, so the fmt gate is byte-identical
in behaviour — verified, its 10-case self-test still passes and it still prints
`4 workspace(s) checked, 4 formatted`. Copying would have put workspace
discovery in two files, and a fact duplicated across copies that then disagree
is exactly how #381 happened.

MEASURED BEFORE WIRING, so this is not a latent red: all four workspaces are
already clippy-clean under `-D warnings` (each run individually, exit 0). Live
gate prints `4 workspace(s) checked, 4 lint-clean.`

The self-test's discriminating case is the runner's exit code — the SAME
four-workspace tree passes when the runner reports clean and FAILS when it
reports dirty. A wrapper that ignored its runner, which is how this gate would
go vacuous, passes the first and fails the second. A third case pins that a
missing cargo is CANNOT_CHECK, not a pass.

(h) THE FUZZ TARGET LIST WAS NEVER COMPARED TO ITS DECLARATION.

`fuzz/Cargo.toml` declares harnesses as `[[bin]]`; `fuzz-nightly.yml` runs them
from a hand-written `matrix.include`. The `include` form is deliberate and
stays — each target carries its own `extra_args`, and only
`fuzz_scheduler_solver` needs `-max_len=128`; a flat list would impose that cap
on two healthy fuzzers (#361). Its cost is two lists maintained separately with
nothing comparing them.

Add a harness and forget the workflow, or drop a matrix entry, and the nightly
stays green having fuzzed a strict subset — no error, no warning, no smaller
number anywhere in the log, because success is reported per leg and nobody
counts the legs. Nightly is where this hides longest: it is not on the PR gate,
and `fuzz_scheduler_solver` was red 97/97 runs from 2026-04-25 before anyone
looked. A silently ABSENT leg is harder to notice than a loudly failing one.

`tools/check_fuzz_targets.py` asserts set equality in BOTH directions, because
the two failures are different bugs: declared-but-not-run is a harness never
exercised; run-but-not-declared is a leg that cannot build and fails looking
like a toolchain problem. Either side coming back EMPTY exits 2 — a parse that
read nothing is a broken scan, not perfect agreement about nothing, which is the
one way this check could itself have gone vacuous. 7-case self-test. Two file
reads, no build.

TWO THINGS THE GATE SURFACED BY BEING RUN, included rather than reverted:

  * `codegen-*/Cargo.lock` pinned `spar-analysis 0.25.0` against a workspace at
    0.35.0 — ten minor versions stale. Nothing routinely built in those
    workspaces, so nothing refreshed them. Updated here; otherwise every
    contributor who runs the new gate gets the same dirty diff.
  * `codegen-*/target/` and `fuzz/target/` were NOT gitignored. `.gitignore:1`
    is `/target`, root-anchored, and the gap never showed because nothing built
    there. It does now. Added.

Verified: all NINE guardrail self-tests pass on this tree; both new gates run
green for real; release-plane and human-scoped gates still pass on the edited
artifacts; workflow and artifact YAML parse.

Refs #383, #390
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Rivet verification gate

20/20 passed

count
Passed 20
Failed 0
Skipped (no steps) 0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.

@avrabe

avrabe commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Red board here is a GitHub Actions outage, not this PR

Recording the diagnosis so nobody re-reads these as code failures. Not one of the non-green checks has executed a single step of real work.

GitHub's own status page, checked independently:

overall: Partial System Outage
  Actions          major_outage
INCIDENT: Incident with Actions [investigating]  2026-08-06T15:22

Every failure on this PR falls inside that window (first attempt 15:10).

The three non-green checks, and why none is a verdict

check conclusion evidence
Lean proof typecheck failure died in Set up job: Failed to resolve action download info. Error: Service Unavailable — before checkout
Code Coverage cancelled no log exists at all (log not found) — never ran a step
Security Audit (RustSec) cancelled same

cancelled is not failure, and both matter differently. The cancellations are self-inflicted and my fault: this workflow sets

concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

so back-to-back gh run rerun --failed across the two runs killed in-flight jobs from the same group. Re-runs were staggered afterwards; the outage outlasted them.

One hypothesis I had, checked and wrong

I suspected the Security Audit red was mine — this PR refreshes codegen-*/Cargo.lock, which plausibly pulls a version carrying an advisory. It is not: the job never reached cargo audit, or any step. Worth stating because acting on that hypothesis would have meant reverting a legitimate fix (those lockfiles pinned spar-analysis 0.25.0 against a workspace at 0.35.0) to chase a failure that never touched the code.

What IS verified, locally

  • 9/9 guardrail self-tests pass on this tree
  • both new gates run green for real — 4 workspace(s) checked, 4 lint-clean and All 3 declared fuzz targets are run
  • all four workspaces measured clippy-clean under -D warnings before wiring, so (g) is not a latent red
  • check_fmt_workspaces behaviourally unchanged by the parameterisation: 10/10 self-test, still 4 workspace(s) checked, 4 formatted

Next

Waiting for the incident to clear, then one clean re-run. Not merging on an outage-red board, and not re-running again while Actions is in major_outage — that just burns runner time on the self-hosted pool.

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@avrabe
avrabe merged commit db6b30d into main Aug 7, 2026
69 of 78 checks passed
@avrabe
avrabe deleted the feat/v036-scope-assertions branch August 7, 2026 00:27
avrabe added a commit that referenced this pull request Aug 7, 2026
#395 (obligations g+h) and this branch (f) both appended a TEST-GUARD-*
artifact at the same insertion point in artifacts/verification.yaml, so the
three-way merge conflicted on adjacency alone. Both sides kept; nothing was
edited or dropped.

Verified on the MERGED tree, not on either parent: 168 artifacts, no
duplicate ids, all three new entries present, check_release_plane /
check_human_scoped / check_required_contexts green, and the four guard
self-tests (lean_sorries 8, clippy_workspaces 3, fuzz_targets 7,
fmt_workspaces) all pass together. Live gates on the merged tree:
"All 3 declared fuzz targets are run" and "12 sorries, at or under the
declared floor of 12".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant