AUD-S13: test-depth bundle (TEST-02/05/06) + coverage floor to 91% - #35
Open
konih wants to merge 7 commits into
Open
AUD-S13: test-depth bundle (TEST-02/05/06) + coverage floor to 91%#35konih wants to merge 7 commits into
konih wants to merge 7 commits into
Conversation
konih
force-pushed
the
lane/aud-s13-test-depth
branch
from
August 8, 2026 13:03
95f55ea to
ee8068f
Compare
…ranches, repo_file containment + expiry TEST-02 (REQ-AUD-S13-01) — internal/core/aggregate/evaluate_tocel_test.go: a json.Number fitting neither int64 nor float64 falls back to its STRING form (evaluate.go:191). Pins that the fallback is observable (type(new) == string) and that every predicate shape over it ERRORS -> fail-safe, plus a Cover-level test proving the error reaches the decision as predicate.error/require-review and never APPROVEs through a string/number confusion. TEST-05 (REQ-AUD-S13-02) — internal/forge/clearslot_test.go: drives forge.Reconcile down the clear-slot path over all branches of reconcileClearSlot, each error branch at both polarities: list failure, resolve refusal, PARTIAL CLEAR (forge reports success but leaves the thread open), idempotent already-clear, rescan-listing failure, no-thread-for-slot, and duplicate open threads. Every refusal asserts zero operations, the thread still open, and an unrelated slot untouched. Error injection uses a local stub embedding *fake.Forge -- no production or fake change. TEST-06 (REQ-AUD-S13-03) — internal/provider/builtin/repo_file_test.go: a path-containment table (traversal, absolute, backslash, normalization-only escape, sibling-prefix directory, walk-up clipping, File-as-basename) and an expiry table (missing/unparseable/zero/negative maxAge; the expiry boundary asserted one nanosecond either side), plus the undecodable/empty-document axis. Rejections assert fact.Value == nil -- never a fact from outside the roots or past expiry. The expiry table asserts the builtin's OWN response as well as the host outcome, because the host schema gate masks the provider reason for a malformed maxAge. Every table carries a case-count positive control so a lost case cannot pass vacuously. Zero production-code changes.
The aggregate ./internal/... total sat at 90.28% against a 90% floor, so the gate had stopped MEASURING and started STEERING: any lane adding a branch without a test reddened task check for a reason unrelated to that lane, and the cheapest green was a filler test. AUD-S13's three behavior families lift it to 91.010% (4479 -> 4515 of 4961 statements), so the floor moves with them. Taskfile.yml coverage: desc, echo, awk threshold and FAIL message all move together -- a half-updated gate prints a message that lies. GUIDELINES.md's '>=90% coverage gate' claim moves with it. Logged as D-128, which also records the two things a future reader needs: the margin is 0.01pp (about half a statement), and .github/workflows/ verify.yaml still enforces 90% because that path belongs to Lane B / PR #25 -- so CI enforces the OLD floor until Lane B raises it. D-128 supersedes the AUD-S13 spec's 'Not in scope: raising the D-010 floor' line; the operator ruled otherwise for this change.
…nership line Two corrections to the row landed one commit ago, both material to how a future reader acts on it. 1. The margin was stated as '0.01pp, about half a statement'. That reads the raw ratio, but the gate compares the ONE-DECIMAL total that `go tool cover -func` prints -- pct is the string '91.0', not '91.010'. Anything rounding to 91.0 passes, so the effective floor is ~90.95% and the true minimum is 4513 of 4961 statements. Actual slack is 2 statements, not half of one. Understating it invites someone to conclude the 91% floor is unsustainable and walk it back. 2. The row assigned the verify.yaml raise to 'Lane B'. PR #25 (AUD-S09/S14), the workflow-owning lane, merged 2026-08-08 -- before this lane landed. No open lane owns .github/workflows/**, so the split-gate residual is UNOWNED, and a reader who trusts the old wording would assume it is covered while CI quietly keeps enforcing 90%. Row text only; no gate behavior change.
…t skew (D-128) The fence that kept AUD-S13 out of .github/workflows/** was PR #25, which merged. So the split gate this lane flagged is now closable, and the right fix is not to raise a second literal -- that only resets the drift clock. verify.yaml no longer re-implements the coverage gate. The 'install Task' step moves above it and the step body is just `task coverage`, so CI runs the same gate developers run. The threshold now exists in exactly one place: Taskfile.yml's coverage: task var COVERAGE_MIN, interpolated into the echo and the awk comparison, with desc naming the var instead of the number. verify.yaml carries no coverage threshold at all any more. Proven at both polarities rather than assumed: COVERAGE_MIN=99 reds the gate with the right message. That control matters specifically because an unrendered {{.COVERAGE_MIN}} would yield min="", min+0 == 0, and a gate that passes everything -- a templating typo would have been a silent fail-open, not a visible break. hack/lint/workflow_pins_test.sh stays green (39 controls), including 'both Task installs interpolate a single workflow-level TASK_VERSION' after the step move; verified the YAML still parses and the step order is right. Also logs the two fail-opens AUD-S13 found while writing its tests, as OQ-27 (toCEL lexical compare returns a silently wrong boolean when BOTH operands are over-range -- and evaldecode.go:61 understates the known residual, so it is a docs-truth defect too) and OQ-28 (repo_file enforces path containment but not filesystem containment; os.DirFS is not a security boundary and the escaping symlink can arrive in the MR under evaluation). Neither is fixed here -- both are routed for a severity ruling. No new D-row: they are undecided, so they are questions, not decisions. D-128 updated: its split-gate residual is closed, and the closure mechanism recorded so a future editor does not re-inline the shell.
… deepen OQ-28 (review F4/F5) Both findings are worse than AUD-S13 first reported. Independent review escalated them; I reproduced each claim before writing it down. OQ-27 was scoped to numeric overflow. The class is ANY string-bound operand under a relational CEL leaf: cel-go defines < <= > >= over two strings, so it returns a clean boolean exactly where the engine assumes an error. The P1 instance needs no overflow at all -- a quoted YAML scalar. Reproduced through the production aggregate.Cover entry point with the D-016-shaped partitions-must-not-shrink rule: 12 -> 6 numeric gives BLOCK with one partition-count-shrunk finding; the identical policy with "12" -> "6" gives APPROVE with ZERO findings. evaldecode keeps !!str a Go string deliberately, but that design assumed the compare would fail safe. Verified no lint guard exists: checkLeafScope and checkPredicateScope validate identifier scope, checkFactsShape validates facts-path shape; none type-checks relational operands. Recorded the severity split explicitly -- mechanism P1, over-range instance alone P2 on reachability, P1 resting on the quoted-string case -- and that hard rule 7 is NOT violated: a lexical compare is deterministic. What is violated is the fail-safe direction and evaldecode's own written claim. Conflating those is how a real finding gets waved away. OQ-28 gains the discriminating question its severity ruling depends on: does internal/change see a symlink swap of a governed path as a change to it? Git records 100644->120000 so enumeration should surface it, but that is asserted not verified, and it separates "reads arbitrary files" from the worse "silently influences a decision". Also recorded: the escape reaches arbitrary ABSOLUTE host paths and has a second, stronger form (a file symlink at a legitimate in-root path, where every inspected path string is clean); it is live in production via provider_host.go's checkoutFS building os.DirFS over the MR HEAD tree, so the symlink arrives in the MR under evaluation; resolved non-sensitive values reach the MR comment through render.formatMessageScalar/displayFactValue, with sensitive:true the only limiter; the target must still be a mapping with a matching top-level key, which keeps it narrow. Named the sibling: LoadResourceOwnerMap reads the same FS with no roots at all, and that registry decides who may approve. No OQ-29 and no D-row: one class, one row, and neither is decided.
…le (review F6) repo_file_test.go's `if fact.Value == float64(99)` could never fire: to reach it a case must already have passed the state assertion and the `fact.Value != nil` check, so Value is nil by construction. It was dead weight that made the table look like it had a guard it did not have. The sibling-prefix case still bites without it -- re-ran the HasPrefix(p, root+"/") -> HasPrefix(p, root) mutation and it fails on `state = "resolved" ... want "unavailable"`. Adjusted the fixture comment to say which assertion actually does the work, and why the per-directory values are still worth keeping (they name the leak's source in the failure output rather than leaving "resolved, but from where?").
konih
force-pushed
the
lane/aud-s13-test-depth
branch
from
August 8, 2026 13:32
cac62b3 to
1cede44
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AUD-S13 — test-depth bundle, tests only. Zero production-code changes (
git diff origin/main...HEAD -- internal/ cmd/ ':!*_test.go'is empty). Three behavior families + the D-010 floor raise the operator ruled on.The three families
TEST-02 —
toCELoverflow semantics (REQ-AUD-S13-01)internal/core/aggregate/evaluate_tocel_test.go(new)A
json.Numberfitting neither int64 nor float64 falls back to its string form (evaluate.go:191). Pinned:type(new) == stringin the activation, andtoCELreturns the literal unchanged;oldside, digit-overflow,int()conversion, arithmetic);TestToCELOverflowNeverApprovesThroughCoverdrives the production entry point (Cover, the real coverage loop) and asserts the error surfaces aspredicate.error/require-review and never APPROVEs — with an in-range control proving the same policy fails differently when the compare is possible.A 60-digit integer is deliberately excluded from the fixture: it overflows int64 but
Float64()succeeds, so it takes the float64 branch, not the fallback.TEST-05 —
reconcileClearSlotbranches (REQ-AUD-S13-02)internal/forge/clearslot_test.go(new)Drives
forge.Reconciledown the clear-slot path over all 8 branches, each error branch at both polarities: list failure, resolve refusal, partial clear (forge reports success but leaves the thread open), idempotent already-clear, rescan-listing failure, no-thread-for-slot, duplicate open threads, and the ordinary single-thread clear.Every refusal asserts three things: zero operations on the receipt, the thread still open, and an unrelated healthy slot untouched. Every success validates the receipt against the frozen publication-receipt schema — which is what makes the idempotent arm's "reference a thread so minItems:1 holds" behavior load-bearing rather than incidental.
Error injection uses a local stub embedding
*fake.Forge— no change to production or to the shared fake. Now confirmed rather than hoped: this branch is rebased onto merged AUD-S15 (#34), which liftedMRInfo/ErrNotFoundinto the forge port, and the stub compiled and passed unchanged.TEST-06 —
repo_filecontainment + expiry (REQ-AUD-S13-03)internal/provider/builtin/repo_file_test.go(extended)"."-root-means-no-clip, sibling-prefix directory, walk-up clipping, empty anchor,File-as-basename (a traversal inFileis basenamed away, not followed),./../emptyFile, nil FS.expiresAt.answerRepoFile's fail-closed list, and the 4 statements that carried the total over 91.0%.The load-bearing assertion on every rejection is
fact.Value == nil— the story's invariant is "never a fact from outside the roots or past expiry", so a rejection still carrying a value is the failure regardless of state label. The fixture gives each directory a distinct value (99 intopics-archive/) so that when a containment check is loosened, the failure output names the leak's source instead of leaving "resolved, but from where?".The expiry table asserts the builtin's own response bytes as well as the host outcome, because for a malformed maxAge the host's schema gate rejects the echoed declaration first and overwrites the reason — asserting only the host outcome would never see which
expiresAtbranch fired.Mutation evidence
Every test was proven RED by breaking the code it covers, confirming the mutation was really in the file via
git diff, then restoring. Nine rounds:toCEL:return x.String()→return float64(0)code="partition-count-shrunk"— i.e. the silent numeric coercion the test forbids)list_error_fails_closed,resolve_error_fails_closedrescanClearSlotcallsalready_clear_rescan_list_error,partial_clear_rescan_mismatchErrUnsupportedDecision→ fabricated receipt; duplicate branch → resolve first; idempotentTargetID→""no_thread_for_slot_unsupported,duplicate_open_threads_fail_closed,already_clear_is_idempotentResolveThreadcallsingle_open_thread_resolved,resolve_error_fails_closedHasPrefix(p, root+"/")→HasPrefix(p, root); dropIsAbs; drop the../escape check; drop the basename check; drop the walk-up clipsibling_prefixfailing onstate = "resolved" … want "unavailable"MaxAge == ""→ dead;d <= 0→d < 0; host!After(now)→Before(now)missing_maxAge,zero_maxAge,exactly_at_expiry_instantd <= 0guardsunparseable_maxAge,negative_maxAge,zero_maxAge!present || val == nil→ deadunparseable_yaml,top_level_sequence,empty_document,null_documentAll 8 clearslot subtests, all 16 containment cases' guards, and all expiry/document error branches were individually shown RED. Every table carries a case-count positive control (
if len(cases) != N) so a lost case cannot pass vacuously, and the clearslot table additionally asserts both polarities are present.Independent review ran six more mutations, two of which I had not — all landed where predicted. Its mutation 2 (
underAnyRoot→return true) showed the anchor check and the walk-up clip are independently pinned; its mutation 4 (deleting the rescan in the case-0 arm only, where I deleted both) showed the two rescan sites are independently isolated.Per review F6 I also removed a dead assertion (
if fact.Value == float64(99)) that could never fire — a case reaching it has already passed the state check andValue != nil. The sibling-prefix case still bites without it; re-verified by re-applying the mutation.Coverage
./internal/...total9e50e17)task coverageprintscoverage: 91.0% (required: 91%). Per-package:internal/forge83.9 → 89.1,internal/provider/builtin76.0 → 85.6,internal/core/aggregate89.7 → 89.9.reconcileClearSlotwent 56.5% → 100%.No filler: every added case asserts a fail-closed outcome. Branches left uncovered are unreachable through the public API (
json.Marshalof a struct that cannot fail,jsonNormalize's error returns,candidates' absolute-path arms,cleanRel'sclean == "/"shadowed byIsAbs) or need a chmod-000 file whose behavior differs under a root CI container.D-row: D-128
Numbered 128 to clear the concurrently-held D-126/D-127; appended only, no existing row touched. It records the floor raise 90% → 91%, that it supersedes the AUD-S13 spec's "Not in scope: raising the D-010 floor" line, and two things a future reader needs:
go tool cover -funcprints (pctis the string91.0, not91.010), so anything rounding to91.0passes and the effective floor is ~90.95%. The true minimum is 4513 of 4961 statements (4513 → 90.9696% → prints91.0→ PASS; 4512 → 90.9494% → prints90.9→ FAIL). This lane leaves 4515, so slack is 2 statements. The rounding is inherited from the pre-existing D-010 gate, not introduced here, and is left as-is.The margin correction landed as a follow-up commit; the row's first version understated it and named a lane that had already shipped.
The coverage floor is now single-sourced — CI and local cannot skew
The floor briefly existed twice:
Taskfile.ymlsaid 91% while.github/workflows/verify.yamlre-implemented the gate inline and still said 90%, because this lane was fenced out of.github/workflows/**while PR #25 owned it. #25 merged and the fence lifted, so rather than raise a second literal — which only resets the drift clock — the CI step now delegates totask coverage:install Taskmoves above the coverage step; the step body is justtask coverage, so CI runs the same gate developers run.verify.yamlcarries no coverage threshold at all any more.Taskfile.yml'scoverage:task varCOVERAGE_MIN: 91, interpolated into the echo and the awk comparison, withdescnaming the var instead of the number.GUIDELINES.mdnamesCOVERAGE_MINas the source instead of silently duplicating the value.Proven at both polarities, not assumed:
COVERAGE_MIN: 99reds the gate with the correct message. That control earns its keep — an unrendered{{.COVERAGE_MIN}}would givemin=""→min+0 == 0→ a gate that passes everything, so a templating typo here would have been a silent fail-open, not a visible break.hack/lint/workflow_pins_test.sh(PR #25's gate) stays green across all 39 controls, including "both Task installs interpolate a single workflow-level TASK_VERSION" after the step move. YAML re-parsed and step order confirmed (install Task 11 → coverage 12 → PCS exit gate still later).Two fail-opens found while writing these tests — OQ-27 / OQ-28. These BLOCK the release tag.
Both are logged in
docs/planning/open-questions.md(amended after review widened them), not left in PR prose. No D-row and no OQ-29 — one class per row, and neither is decided. Neither is fixed here: tests-only lane, and both fixes need their own lanes (the decision-path one must not be bundled with anything).OQ-27 (P1) — a relational CEL leaf over STRING-bound operands returns a silently wrong boolean. Verified BLOCK → APPROVE flip. I originally scoped this to numeric overflow; review F4 correctly widened it to any string-bound operand, and the widened case needs no overflow at all. cel-go defines
< <= > >=over two strings, so it returns a clean boolean exactly where the engine assumes an error.I reproduced the P1 case through the production
aggregate.Coverentry point before writing it down — same D-016-shapedpartitions-must-not-shrinkrule, same subject:partitions: 12 → 6(numeric)partition-count-shrunk)partitions: "12" → "6"(quoted)evalLeafreturns(true, nil), the obligation records as proven, and a destructive change auto-merges.evaldecodekeeps a!!stra Go string deliberately — but that design assumed the compare would fail safe. Also verified: no lint guard exists (checkLeafScope/checkPredicateScopedo identifier scope,checkFactsShapedoes facts-path shape; none types relational operands). The over-range case is the same mechanism, second instance.Severity split, kept explicit: mechanism P1; over-range instance alone P2 on reachability (needs both operands > ~1.8e308); the P1 rests on the quoted-string case. Hard rule 7 is NOT violated — a lexical compare is deterministic; what's violated is the fail-safe direction and
evaldecode.go:61's own written claim (it calls this "ADR-0013 residual #1" but describes only a "lossy compare", never a silently wrong boolean — a docs-truth defect too). I deliberately wrote no test blessing eithertrue.OQ-28 —
repo_fileenforces path containment but not filesystem containment; it reads arbitrary absolute host paths, live in production.os.DirFSis explicitly documented in Go as not a security boundary; path-level and filesystem-level containment are different guarantees and only the former is held. Review F5 found a stronger second form than mine: not just a directory symlink (topics/evil -> <outside>) but a file symlink at a wholly legitimate in-root path —topics/prod/quota.yaml -> /abs/outside/cluster-secrets.yaml— where every path string the guards inspect is impeccable, and the target is an arbitrary absolute host path.It is live in production, not test-only:
cmd/assent/provider_host.go'scheckoutFS()buildsos.DirFS(<checkoutRoot>/head)— the MR head tree, i.e. attacker-authored content — reachable viaassent run --checkout. So the symlink arrives in the MR under evaluation (I verified this call path).Exfiltration seam: a resolved non-sensitive fact value reaches the forge-facing comment via
render.formatMessageScalar→displayFactValue, so a rulemessageinterpolating a fact value echoes the file's contents into the MR comment;sensitive: truerenders[redacted]and is the only limiter. Constraint that keeps it narrow (stated so the finding isn't overstated): the target must be a YAML/JSON mapping with a top-level key matching a declared output name.Discriminating question the severity ruling needs — does
internal/changesee a symlink swap of a governed path as a change to that path? Git records100644→120000, so enumeration should surface it, but that is asserted, not verified, and it separates "reads arbitrary mapping-shaped files" from the worse "silently influences a decision".Sibling with worse blast radius:
LoadResourceOwnerMap(resource_owner.go:93) does a barefs.ReadFileover the same FS with no roots at all — and that registry decides who may approve (verified). Fix:os.OpenRoot()+(*os.Root).FS(), confirmed available ongo 1.25.0.Gates
On
main@e54a243(includes #27 exec-flake fix and #34 AUD-S15 forge-port lift), head1cede44. On a completed run — graded on stage banners, not just exit code:task checkEXIT=0, all 14 stage banners present (fmt vet lint test coverage build dogfood-comparison compare-exitgate-test changelog-verify release-changelog-gate-test release-verify-tag-gate-test docs-gates lint-depguard-test lint-workflow-pins-test), 0 failures,coverage: 91.0% (required: 91%).task determinismEXIT=0.hack/lint/workflow_pins_test.shEXIT=0, 39 controls.task changelog-writeproduces no diff;changelog-verifygreen insidecheck).The
TestExecDigestPin/hack/spikes/providersignal: killedflake that aborted 3 of 5 earlier runs is gone now that #27 is in the base.Named verify commands, all executing non-vacuously (
-vconfirmed):go test ./internal/core/aggregate/... -run TestToCELOverflowFailsSafego test ./internal/forge/... -run TestReconcileClearSlotBranchesgo test ./internal/provider/builtin/... -run 'TestRepoFileContainment|TestRepoFileExpiry'task coverage→ 91.0% ≥ 91.0Files
Tests (the deliverable):
internal/core/aggregate/evaluate_tocel_test.go(new) ·internal/forge/clearslot_test.go(new) ·internal/provider/builtin/repo_file_test.goGate:
Taskfile.yml(COVERAGE_MIN) ·.github/workflows/verify.yaml(delegates totask coverage) ·GUIDELINES.mdRecord:
docs/decisions/decisions.md(D-128) ·docs/planning/open-questions.md(OQ-27, OQ-28) ·CHANGELOG.mdStill zero production Go changes —
git diff origin/main...HEAD -- internal/ cmd/ ':!*_test.go'is empty.