test(opencode): pin every permission rule individually β 39 of 77 deletions were invisible to the gate - #352
Merged
Conversation
β¦etions were invisible PR #348 restored 52 `ask` values a blind sed had flattened. An adversarial audit then found it had closed two INSTANCES of a coverage class, not the class: a permission rule could be deleted outright with the suite fully green, while a realistic command in that family dropped to plain `allow` at BOTH the glob layer and the layered (glob + guard_core) layer. MEASURED here independently, not taken from the audit β delete each of opencode.jsonc's 77 single-line verdict rows one at a time, run test_opencode_config.py under the FLAKE-PINNED opencode 1.18.4 (an ad-hoc nix-shell serves 1.18.11 and fabricates ~15 failures), attribute by failing test id: before: 39 / 77 deletions left the suite GREEN after: 10 / 77 β and all ten are non-bash tool rows Of the 39, 17 were real holes, one more than the audit found: it missed the anchored `sudo*`, whose sole unique coverage is a `sudo` NOT followed by a space (`sudoedit`). That is the exact mirror of the `*sudo *` finding the audit's own MUST_ASK comment records having found by mutation testing. The second defect was the aggregate floor. `len(asks) >= 40` against 50 measured left ten rules of slack, and the docstring called it "room to prune ten deliberately" without noticing that is the same thing as room to un-gate ten dangerous families. Reproduced: deleting the four mutating `systemctl` verbs, kubectl patch/scale/drain, helm upgrade/rollback and flux resume leaves exactly 40 and reports 480 passed, 0 failed while all ten resolve `glob=allow layered=allow`. WHAT CHANGED (tests + comments only β no permission rule is altered; the config was already correct, the suite could not tell): * 18 MUST_ASK rows, one per hole, each measured to be pinned by exactly one rule so it kills that rule's deletion mutant and no other. * `test_every_ask_rule_is_individually_pinned` / `..._deny_rule_...` β the deletion sweep as a standing assertion. Each rule must be the SOLE DECIDER for a pinned command, or be declared redundant / guard-backstopped, with the declaration RE-MEASURED so it cannot outlive the rule it depends on. These fail when the rule set grows AND when it shrinks. * floor raised 40 -> 50, the measured count, with no cushion. A deliberate prune must now lower the literal in the same commit. * `bash_ruleset()` memoised (immutable tuple): the ledger resolves ~63 rules x ~90 commands and took the file 1.7s -> 18.5s. `load_config()` stays uncached because test_opencode_engine.py mutates its result in place. `rm -r`: the premise that guard_core "only handles `-rf` forms" is FALSE, measured. check_rm_rf_critical tests the recursive flag alone (`--recursive`, or any bundle containing r/R) and never requires `-f`; `rm -r /`, `rm --recursive $HOME`, `rm -r .` and `rm -r /etc` are all denied today. No guard_core change. Its non-coverage of an ORDINARY path (`rm -r /repo/build`) is a documented decision, so that case is pinned in MUST_ASK β the glob is its only gate β and the attribution is pinned by a test labelled as the invariant guard it is. VERIFIED against the real engine, not a reading of the resolver: all 18 new pins resolve `ask` under `opencode debug agent build --pure` on the flake-pinned 1.18.4, and in each case the intended rule is the LAST matching one, so none is shadowed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
β¦y-downgrade mutant
Follow-on within the same PR, from re-running my own sweep against the fix
rather than stopping at the headline number:
* 10 of 77 deletion mutants still survived, and ALL TEN were the non-bash
tool-permission rows. `doom_loop` and `external_directory` are the ones that
matter β the only `ask` rows outside the bash block, and flipping either to
`allow` removed a real control with the suite green.
`test_tool_level_permissions_are_pinned_exactly` pins them as an exact map,
which fails on delete, comment-out, verdict-flip AND an added row.
* one `flip-ask` mutant survived everything: `"*git*add -A*"` downgraded
deny -> ask. Invisible end-to-end because guard_core denies `git add -A`
anyway β but NOT an equivalent mutant, since the deny block exists for the
case where guard.js fails to load, where the downgrade would offer a human
the chance to approve it in the TUI.
Fixing that exposed the shape of the bug in my own first attempt: reading
`denies` off the config and checking only what is IN it cannot see a rule
that LEFT the set. The deny ledger now pins the SET first, against declared
GLOB_ENFORCED_DENIES / GUARD_BACKSTOPPED_DENIES.
* removed the deny ledger's "stale declaration" assertion: the new set check
subsumes it, so it had become UNREACHABLE β an assertion that can never
execute reports safety while testing nothing. Verified by mutation (dropping
`*mkswap*` fails on the set assertion, never reaching it).
Mutation results after this commit, all under the flake-pinned opencode 1.18.4:
delete 0 / 77 survive (39 before)
comment-out 0 / 77 (39 before)
flip->allow 0 / 68 ( 2 before)
flip->ask 0 / 16 ( 1 before)
reorder 0 / 63 ( 0 before)
All 9 new assertion branches proven individually REACHABLE, each failing with
its OWN message β not merely "I broke it and something went red".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
β¦mission-coverage
β¦mission-coverage
β¦ close the sudoedit gap Audit round on #352. F1/F2/F4; F3 and most of F8 deferred to their own PR. F1 β THREE OF SIX REDUNDANCY DECLARATIONS WERE FALSE BY MEASUREMENT. Re-measured on the model AND the real 1.18.4 engine, deleting each glob: *nix-collect-garbage* `nix-collect-garbage`, `β¦ --max-freed 1G` ask -> allow *dd if=* `FOO=1 dd if=/dev/sda`, `time dd if=β¦` ask -> allow *dd of=* `FOO=1 dd of=/tmp/img bs=1M` ask -> allow `*age*-d*` rescues only spellings containing a literal `-d`; `dd *` is anchored, so every prefixed dd rested on the two operand rules alone. All three are now pinned in MUST_ASK. The jsonc comment this PR had ADDED asserting "MEASURED: this rule is REDUNDANT β deleting it changes no verdict" was false and is corrected: a comment is a claim, and that one invited deleting a live gate. The other three declarations are genuinely subsumed β verified, kept. F2 β THE DECLARATION WAS AN ESCAPE HATCH, and F1 is that failure mode already shipped. It mapped a pattern to PROSE, so the ledger could only check "no pinned command moved" β i.e. it verified entries against the pinned pool, never against the claim. Reproduced with controls: adding `"*terraform*destroy*": "ask"` went RED, but declaring it covered by a pattern that DOES NOT EXIST went GREEN at 574. Now `pattern -> (covering_pattern, witness_command)`, with all three legs re-measured: the coverer exists, the witness is really matched by the redundant rule, and with that rule removed the witness is last-matched by exactly the declared coverer. Imaginary, wrong-but-real, and non-matching-witness variants are all RED (4 controls). π΄ F2 ALSO EXPOSED THIS PR'S OWN DEFECT CLASS IN THIS PR'S OWN FIX. Extending that loop dropped the `elif not killers: unpinned.append(pat)` branch, so `unpinned` β the assertion this whole file exists for β became DEAD CODE while the suite reported 616 passed and the entire mutation battery stayed at 0 survivors (the deleted rules were still caught by the per-command assertions). Nothing could see it, because the ledger's reassuring answer is an empty list and an empty list is what a ledger wired to nothing returns. Fixed, and made unrepeatable: `test_an_unpinned_ask_rule_is_detected` is a standing POSITIVE control that feeds the ledger a synthetic unpinned rule and asserts the list moves. The ledger body is now one helper both call, so there is no second copy. F4a β `*sudoedit*` added to the config. `sudo*` and `*sudo *` were presented as complementary halves; their INTERSECTION was covered by neither. MEASURED ALLOW at both layers before this: `FOO=1 sudoedit β¦`, `EDITOR=vim sudoedit β¦` (the NORMAL invocation β sudoedit reads $EDITOR) and `/run/wrappers/bin/sudoedit β¦`. Unlike a flag spelling, `sudoedit` is a fixed binary name, so one leading-and- infix `*` closes the family rather than buying one more spelling. `sudo*` is re-pinned via `sudo-rs whoami` β a bare `sudo` does NOT isolate it, because `"*sudo *"`'s `" *"` suffix also matches the bare form. F4b β the `rm` comment claimed the glob was "the ONLY thing between an agent and a silent recursive delete". It is not even that: `rm -R`, `rm -Rf` and `rm -fr` on an ordinary path resolve ALLOW at BOTH layers. Comment corrected and the gap PINNED as an open, labelled characterization test rather than closed, because widening the glob is whack-a-mole (`-R`, then `-fR`, then `-vr`) and a candidate that looked general β `"*rm*-*r*"` β matched `rm -R /repo/build` only because the PATH contains an "r", so it fails on `rm -R /x`. The structural fix is guard_core's target set, which its own docstring reserves as an operator decision needing real measurement. F8 (partial) β the systemctl comment read as exhaustive. MEASURED allow at both layers: mask, unmask, kill, isolate, set-property, edit, reload. Scope corrected; the wider verb sweep stays deferred. Floor 50 -> 51, per this PR's own no-cushion policy, because `*sudoedit*` landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
β¦mission-coverage
β¦mission-coverage
Delta-audit round on #352. N1, N2, N4, N5 fixed; N3 softened to match the code; N6/N7/N8 folded in where they were a line; N9 recorded in the PR body. N2 β `time` DEFEATED BOTH LAYERS AT ONCE. time dd bs=1M if=/dev/zero of=/dev/sda -> glob=allow guard=allow `time` was absent from `_WRAPPER_VALUE_FLAGS`, so argv[0] stayed "time" and check_dd_to_block_device bailed at `basename(argv[0]) != "dd"`. Added, with GNU time's value-taking flags. `\time` needs no entry (the tokeniser already normalises it) and `/usr/bin/time` is handled by the basename β both pinned. π΄ WHY dd, and the part worth keeping: for every OTHER family an unpeeled prefix is still caught by the GLOB layer β measured, `time rm -rf /`, `time mkfsβ¦`, `time talosctl β¦ reset`, `time git stash` all deny on the globs alone. The dd globs are the only SPELLED ones (`dd *` anchored, `*dd if=*`/`*dd of=*` literal), so dd is the one family where an unknown wrapper defeats both. The comment this PR had added told a maintainer the guard covered it; corrected. I also swept the wrapper class rather than fixing the one instance: 14 more prefixes are unpeeled (`xargs`, `watch`, `script -c`, `unbuffer`, `strace -f`, `ltrace`, `catchsegv`, `proxychains`, `torify`, `flock <f>`, `eatmydata`, `systemd-run`, `pv`, `rlwrap`) and each still defeats both layers on dd. NOT added here: guard_core's own header says a wrong flag-arity entry FAILS OPEN, and several of those take positionals or a command STRING (`flock <file> cmd`, `script -c "β¦"`, `systemd-run --unit=x`). That is a table-design job with its own evidence, not a same-round guess. Recorded in the PR body. N1 β MY OWN THESIS, APPLIED TO THE DETECTORS I ADDED. Every ledger detector reports health as an EMPTY LIST, which is exactly what a detector wired to nothing returns. Neutering any one of them left the file 617/617 GREEN β including the three `bad_claim` legs this PR had just added, for which I wrote a control covering only one dimension. Both ledgers are now extracted into `_ask_ledger`/`_deny_ledger` over an EXPLICIT ruleset, and `test_ask_ledger_detector_is_wired` / `test_deny_ledger_detector_is_wired` drive each detector to non-empty with a case built for it β reported as a PAIR (clean empty, seeded non-empty) and, for the three legs, asserting WHICH leg fired so one cannot preempt another. MEASURED after: all 12 detector-neutering mutants go RED (12, not 9 β the extraction split deny's set check into missing/undeclared and made each `bad_claim` leg separately mutable). N4 β the ledger carried a private copy of the resolution loop while the deny side used `effective_bash_action`. Two copies, no disagreement today, nothing pinning that. Collapsed onto one `_resolve_over(rules, command)` + `_rules_without(rules, drop)`; every resolver in the file now routes through it. N5 β the floor's paragraph claimed "the literal is still raised when one lands, so the number never drifts back into being a cushion". That was prose. MEASURED: adding an ask rule AND pinning it, with the literal left at 51, was GREEN under `>=` β the cushion regrows silently until it is the same slack the same docstring spends two paragraphs condemning. Now `== 51`, with a message naming both directions. N3 (deferred fix, docstring softened) β the redundancy hatch is NARROWED, not closed: the witness is declarer-chosen, so the check establishes "another rule wins on THIS command" while the claim is "another rule covers the FAMILY". Nothing verifies subsumption. Said so where the dict is defined. N8/N6 β the hard-coded `*terraform*destroy*` control is gone, replaced by ASK_CONTROLS[0] over a guaranteed-absent SENTINEL plus a test asserting the sentinel really is absent. Perf: the controls made this CI-gated file 46 s. `wildcard_match` is memoised (pure function; the ledgers resolve ~50 rules x ~115 commands x ~65 patterns per pass) and the clean pass is computed once. 3.5 s with 628 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
β¦mission-coverage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The class, not two more instances
#348 restored 52
askvalues a blind sed had flattened toallow. An adversarial audit then found it had closed two instances of a coverage class, not the class itself: a permission rule could be deleted outright with the suite fully green, while a realistic command in that family dropped to plainallowat both the glob layer and the layered (glob +guard_core) layer.This PR closes the class. No permission rule is changed β the config was already correct; the suite could not tell.
(a) Deletion sweep β independently constructed, before/after
I did not trust the audit's list. I re-derived the rule set, re-ran the sweep, and report my own numbers.
Scope, re-derived: the audit's "77 rule lines" = every single-line
"pattern": "verdict"row in the file. I confirmed the decomposition rather than assuming it: 64 in the bash block (1allow+ 50ask+ 13deny) + 13 outside it (10 top-level tool permissions + 3 single-lineplan-agent rows) = 77.Method: delete one row, repair the trailing comma, run
test_opencode_config.py, attribute the kill by failing test id. Counts parsed from pytest's summary line β never an exit code.allowdenyβaskallow)Controls (a sweep with no control is a fact about the sweep):
*nixos-rebuild*must go red: RED on both trees (4 failures on main, 5 on HEAD).Of the 39 survivors, 17 were real holes β one more than the audit found. A survivor only counts as a hole if a realistic command resolves
allowat both layers once the rule is gone; the rest are redundant or guard-backstopped. The extra hole:That is the exact mirror of the
*sudo *finding the audit's ownMUST_ASKcomment records having found by mutation testing β the same class, the other half, which is why the class needed closing rather than the instances.I also confirmed the audit's non-holes rather than taking them on trust:
*sops*--decrypt*,*age*--decrypt*,*systemctl*restart*,*nix-collect-garbage*are redundantly covered, and*git*add --all*/*git*add ./*git*clean -f*/*talosctl*reset*/*mke2fs*/*mkswap*are guard-backstopped by design. Two the audit listed as holes are not:*dd if=*and*dd of=*staydenyviaguard_coreeven with the glob gone.(b) Per-family coverage
18
MUST_ASKrows, one per hole, in the house spellings that RULES.md / k8s.md actually mandate (git -C <path> β¦,KUBECONFIG=$KC_β¦ kubectl -n ns β¦) β i.e. the habitual spelling is the bypassing one. Each is measured to be decided by exactly one rule, so it kills that rule's mutant and no other.But 18 hand-written rows rot, and the next rule added would be unpinned again. So the real fix is the sweep as a standing assertion:
test_every_ask_rule_is_individually_pinnedβ everyaskrule must be the sole decider for some pinned command, or be declared inREDUNDANTLY_COVERED_ASKSwith the covering pattern named. The declaration is re-measured, not trusted: an entry that turns out to be load-bearing fails, and so does a stale one. It fails when the rule set grows and when it shrinks.test_every_deny_rule_is_individually_pinnedβ the deny half, asserted at the layer that actually holds it. Guard-backstopped entries are re-measured to still resolvedenywith the glob removed, so it fails if someone deletes the guard check believing the glob has it, or vice versa.test_tool_level_permissions_are_pinned_exactlyβ the 10 non-bash rows, which were the entire residue after the bash ledger landed.doom_loop/external_directoryare the ones that matter: the onlyaskrows outside the bash block, and flipping either toallowremoved a real control with the suite green.Two findings from re-running the sweep against my own fix (the reason the "after" column is 0 and not 10):
denyβaskdowngrade of*git*add -A*survived everything, becauseguard_coredeniesgit add -Aanyway. Not an equivalent mutant: the deny block exists precisely for the case whereguard.jsfails to load, where the downgrade would let a human approve it in the TUI. Fixing it exposed the same bug in my own first attempt β readingdeniesoff the config and checking only what is in it cannot see a rule that left the set. The ledger now pins the set first, against declaredGLOB_ENFORCED_DENIES/GUARD_BACKSTOPPED_DENIES.(c) The
>= 40floor β decision: raised to 50, the measured count, with no cushionThe slack was the whole defect. Reproduced exactly, and this is the headline:
The docstring called 40 "room to prune ten deliberately" without noticing that is the same slack as room to un-gate ten dangerous families.
Why raise rather than delete it. Its unique coverage is now nearly nil and I say so in the docstring: a mass
askβallowflip is already caught bytest_no_allow_rule_follows_the_wildcard(that produced the original 117 failures),askβdenyby the ordering assertion, and mass deletion β its one former unique catch β is now caught rule-by-rule and by name by the ledger. What it still buys: it keepstest_all_asks_precede_all_denies's ordering assertion and the prefix-tolerance test non-vacuous (the documented reason it exists), and it fails on the count, localising a mass edit to one obvious assertion before the ledger enumerates it.The trade, stated: a deliberate prune must now lower the literal in the same commit. That is the point β it makes a prune a visible act instead of free slack β and it is not extra ceremony, since a prune already requires removing the rule's
MUST_ASKrow.(d)
rm -rβ the premise is false; noguard_corechangeThe brief held that
guard_coreonly handles-rfforms. Measured, not read:check_rm_rf_criticaltests the recursive flag alone β--recursive, or any short bundle containingr/Rβ and never requires-f:rm -r /rm --recursive $HOMErm -r ./rm -r /etc/rm --recursive /nixrm -R //rm -fr /rm -r /repo/buildSo there was nothing to fix, and no behaviour change β which also keeps this PR out of
scripts/claude-hooks/**.What is real is narrower:
guard_coredeliberately does not fire on an ordinary path (its fatal-target set is/,~/$HOME,./.., top-level system dirs), documented as a decision so routine cleanup does not train the operator to route around the guard. So forrm -r /repo/buildthe glob is the only gate β and nothing pinned it. That is now twoMUST_ASKrows (regression coverage), plustest_guard_core_catches_recursive_rm_without_the_force_flag, which is labelled in its own docstring as an invariant guard and is not counted as regression coverage β it exists so the false premise cannot be re-derived into a needless widening.(e) Red/green matrix
This is a coverage fix β
opencode.jsonccarries only comments β so "pre-change code" is the pre-change test data, and each regression gets its own isolation.MUST_ASKrows removed*nixos-rebuild*deletedBranch reachability β "a test failed" is not enough. A mutant can go red via an earlier assert while the branch you care about never runs. All 9 new assertion branches were individually reached with a mutant chosen for that branch, and each was confirmed to fail with its own message: ask/unpinned, ask/wrongly-redundant, ask/stale, deny/unpinned, deny/unheld, deny/set, deny/downgraded, the floor, and the
rmattribution. (This is how I found the deadstalebranch in (b).)Verification mechanism β stated per number
π΄ The version trap was avoided deliberately. An ad-hoc
nix-shell -p opencodeserves 1.18.11 while the tests pin 1.18.4, fabricating ~15test_opencode_engine.pyfailures. Every number here comes from one of three mechanisms, all resolvingpkgs.opencodefrom this flake's nixpkgs:nix build .#checks.x86_64-linux.pytests(nix sandbox) β the authoritative gate. Source of the totals below.buildEnvbuilt from this flake's nixpkgs β used for the 77Γ5 mutation sweeps (a full gate run per mutant was not affordable).opencode --versionasserted 1.18.4 before use.opencode debug agent build --purefrom the same pinned binary β the real engine.Engine verification (3): all 18 new pins resolve
askagainst the real engine, and in each case the intended rule is the LAST matching one β none is shadowed. Gotcha recorded: the engine's stdout truncated at exactly 8192 bytes, nondeterministically, when read through a pipe; I write it to a file (9399 bytes) so "changed CLI" and "full buffer" stay distinguishable.Gate totals (1) β merged tree, per RULES "gate on the MERGED tree":
origin/main@476ed51(built standalone as the control)origin/mainThe gate is RED on both β for pre-existing reasons this PR does not touch, attributed by CAUSE:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/env'inside the nix sandbox (all 27 tracebacks show that identical error). Split by file as I measured it: 11 intest_monitor_blackout.py, 16 intest_rig_control.pyβ the two sum exactly to the 27 distinct failing test functions. (I was briefed 22/5; my measurement disagrees on the split. It does not affect the conclusion β one cause, pre-existing, unrelated.)main, not from this PR β flagged because someone should fix it:test_live_scraper_observes_the_real_config(scripts/collector/keylog/tests/test_espanso_detect.py:544) asserts":date" in trigs, but espanso: prune :date/:time/:uuid + drop the non-uniqueframewossh term; raise cpu-mon thresholdsΒ #351 pruned the:dateespanso snippet fromnix/home.nixwithout updating it. Confirmed pre-existing by the discriminating control above βorigin/mainalone fails it too.datapacket-talos clone not present) β also pre-existing onmain.No absolute green claim is made. The claim is the delta: +94 tests, +94 passing, +0 failures.
Also in here
bash_ruleset()is memoised as an immutable tuple. The ledger resolves ~63 rules Γ ~90 commands and took this CI-gated file from 1.7s β 18.5s; it is back to ~1.7s. A tuple because a cached mutable result is poisonable, and this repo has a caller that mutates a parsed config in place (test_opencode_engine.py'sadd_trailing_git_allow) β which is exactly whyload_config()is left uncached.Two comments added to
opencode.jsonc(no rule changes): that*nix-collect-garbage*is covered only by accident β*age*-d*matching "nix-collect-garbage β¦ -d" β so nobody deletes*age*-d*thinking GC is separately gated; and that bothsudorows are load-bearing.π€ Generated with Claude Code
Audit round 2 β F1, F2, F4 fixed; F3 and most of F8 deferred
The blind audit reproduced every headline number with its own generator. It also found that three of the six redundancy declarations this PR introduced were false by measurement β i.e. the PR's own defect class, in the PR's own new artifact. All three findings are fixed. This round reset the gate: the full battery, the reachability sweep, the engine check and the merged-tree gate were all re-run from scratch, not just for what changed.
π΄ The finding that matters most is one the audit did not report β my own fix had gone dead
While building the F2 control I found that extending the redundancy loop had dropped the
elif not killers: unpinned.append(pat)branch.unpinnedβ the assertion this entire PR exists for β was dead code.It was invisible to everything:
The mutation battery could not see it because the rules it deletes were still caught by the per-command assertions; only a rule that no pinned command matches exposes it. The ledger's reassuring answer is an empty list, and an empty list is exactly what a ledger wired to nothing returns β the "positive control" hazard, one level up from where I had applied it.
Fixed, and made unrepeatable:
test_an_unpinned_ask_rule_is_detectedis now a standing positive control that feeds the ledger a synthetic unpinned rule and asserts the list moves (([], ['*terraform*destroy*'])). The ledger body is extracted into one helper both the control and the real test call, so there is no second copy to drift.F1 β three declarations were false. Measured, at both layers, on model and real engine
allowwith it deleted*nix-collect-garbage*nix-collect-garbage,β¦ --max-freed 1G*dd if=*FOO=1 dd if=/dev/sda,time dd if=/dev/zero bs=1M count=100*dd of=*FOO=1 dd of=/tmp/img bs=1M*age*-d*rescues only spellings containing a literal-d;dd *is anchored, so every prefixedddrested on the two operand rules alone. All are now pinned inMUST_ASK.The jsonc comment this PR added β "MEASURED: this rule is REDUNDANT β deleting it changes no verdict" β was false and is corrected. That is the worst kind of wrong comment: it invites a maintainer to delete a live gate. The other three declarations (
*sops*--decrypt*,*age*--decrypt*,*systemctl*restart*) I re-verified as genuinely subsumed and kept.F2 β the declaration was an escape hatch; F1 was that hatch already used
It mapped a pattern to prose, so the ledger could only check "no pinned command moved" β verifying entries against the pinned pool, never against the claim. Now:
with all three legs re-measured: the coverer exists, the witness really is matched by the redundant rule, and with that rule removed the witness is last-matched by exactly the declared coverer.
Controls (
test_opencode_config.pyonly; direct pytest, flake-pinned 1.18.4):"*terraform*destroy*": "ask"F4a β
*sudoedit*added to the configsudo*and*sudo *were presented as complementary halves; their intersection was covered by neither. Measured ALLOW at both layers before this rule:FOO=1 sudoedit /etc/nixos/configuration.nixEDITOR=vim sudoedit /etc/nixos/configuration.nixβ the normal invocation, since sudoedit reads$EDITOR/run/wrappers/bin/sudoedit /etc/nixos/configuration.nixβ the NixOS pathUnlike a flag spelling,
sudoeditis a fixed binary name, so one leading-and-infix*closes the family rather than buying one more spelling.sudo*is re-pinned viasudo-rs whoamiβ note a baresudodoes not isolate it, because"*sudo *"'s" *"suffix also matches the bare form.F4b β
rm: comment corrected, gap pinned open, not closedThe comment claimed the glob was "the ONLY thing between an agent and a silent recursive delete". It is not even that β
rm -R,rm -Rf,rm -fron an ordinary path are ALLOW at both layers.I did not widen the glob, deliberately: it is whack-a-mole (
-R, then-fR, then-vr), and the candidate that looked like it generalised,"*rm*-*r*", matchedrm -R /repo/buildonly because the path contains an "r" β it fails onrm -R /x. The structural fix is guard_core's target set, which its own docstring reserves as an operator decision needing real measurement, and which is outside this PR's remit.So the gap is pinned open by
test_rm_glob_misses_these_recursive_spellingsβ labelled a characterization test, asserting today's wrong answer on purpose, and telling you to move the row intoMUST_ASKif you close it. Green before and after; not counted as regression coverage.F8 (partial) β comment scope corrected
The systemctl comment read as exhaustive. Measured ALLOW at both layers:
mask,unmask,kill,isolate,set-property,edit,reload. Comment now says so explicitly. The wider verb sweep stays deferred, as agreed.Floor 50 β 51
*sudoedit*landed, so by this PR's own no-cushion policy the literal moves with it. That is the policy working, not ceremony.Re-verification after the fix round β everything re-run
Mutant battery (mechanism: direct pytest under a
buildEnvfrom this flake's nixpkgs;opencode --versionasserted 1.18.4; 78 rows now,*sudoedit*added):allowdenyβaskBranch reachability: 12 / 12, each failing with its own message β now including the two new
bad_claimlegs and the positive control. Two ordering facts worth recording: thewrongly_redundantbranch had to be re-reached with a different mutant becausebad_claimnow legitimately fires first, and the earlier round's deadstalebranch stayed removed.Real engine (
opencode debug agent build --pure, flake-pinned 1.18.4, stdout read from a file β 9493 bytes; the 8192-byte pipe truncation is real, pre-existing attest_opencode_engine.py:175, and not repeated here): all 26 pins resolveask, and in each case the intended rule is the LAST matching one β none shadowed.Merged-tree gate (mechanism:
nix build .#checks.x86_64-linux.pytests; control built standalone from currentorigin/main):origin/main@0cbc9e6The failing sets are identical β the set difference is empty in both directions, so nothing here introduces or masks a failure. Both trees are red for the same pre-existing reasons: 27 failures from one cause (
/usr/bin/envabsent in the nix sandbox; my split, 11test_monitor_blackout.py+ 16test_rig_control.py, sums exactly to the 27 distinct functions β the audit confirmed this over the 22/5 I was briefed), plustest_live_scraper_observes_the_real_config, still broken onmainby #351's:dateprune, plus 2 unpinned skip groups.No absolute green claim. The claim is the delta: +137 tests, +137 passing, +0 failures.
Delta-audit round 3 β N1, N2, N4, N5 fixed; N3 softened; N9 recorded
Re-baselined:
mainis green now. Every earlier statement in this PR about apre-existing red gate (27
/usr/bin/envfailures, the:dateregression, unpinnedskip groups) is superseded β #354β#357 and #350 fixed them. Nothing below is
attributed to a pre-existing failure, because there are none.
π΄ N2 β
timedefeated BOTH layers at oncetimewas missing from_WRAPPER_VALUE_FLAGS, soargv[0]stayed"time"andcheck_dd_to_block_devicebailed at itsbasename(argv[0]) != "dd". Fixed structurally βtimeadded with GNU time's value-taking flags (-f/--format,-o/--output).\timeneeds no entry (the tokeniser already normalises it) and/usr/bin/timeis covered by thebasename(); both are pinned anyway.And the comment this PR had added said the guard covered it. Corrected β that is the second time this PR shipped a comment asserting a measurement that did not hold, and both times the comment was the part that would have caused harm.
Why
ddand nothing else β worth keeping, because it explains the shape rather than the instance: for every other family an unpeeled prefix is still caught by the glob layer. Measured:time rm -rf /,time mkfs.ext4 β¦,time talosctl β¦ reset,time git stashall still resolve deny on the globs alone. The dd globs are the only spelled ones (dd *anchored,*dd if=*/*dd of=*literal), so dd is the single family where an unrecognised wrapper defeats both layers together.I swept the class rather than fixing the instance. 14 more prefixes are unpeeled and each still defeats both layers on dd:
xargs,watch,script -c,unbuffer,strace -f,ltrace,catchsegv,proxychains,torify,flock <file>,eatmydata,systemd-run,pv,rlwrapDeliberately NOT added in this round. guard_core's own header records that a wrong flag-arity entry fails OPEN, and several of these take positionals or a command string (
flock <file> cmd,script -c "β¦",systemd-run --unit=x cmd). That is a table-design job needing its own evidence, not a same-round guess. Follow-up.Tests: 12
timespellings + 5 other-family spellings, RED before the table entry (16 failures), GREEN after. Two further tests β thattime ls/time git statusstay allowed, and that a baretimedoes not crash β are green both ways and labelled invariant guards; without them "the guard got stricter" and "the guard got broken" look identical from the green side.π΄ N1 β my own thesis, applied to the detectors I had just added
Every ledger detector reports health as an empty list β the shape a detector wired to nothing also produces. Neutering any one left the file 617/617 GREEN, including the three
bad_claimlegs this PR introduced, for which I had written a control covering only therulesdimension and not thedeclarationsone.Both ledgers are now extracted into
_ask_ledger/_deny_ledgerover an explicit ruleset, and two parametrized controls drive each detector to non-empty with a case built for it β reported as a pair (clean empty, seeded non-empty), and for the three legs asserting which leg fired so one cannot silently preempt another.Measured after: all 12 detector-neutering mutants go RED. Twelve, not nine β the extraction split the deny set-check into
missing/undeclaredand made eachbad_claimleg separately mutable.unpinned,wrongly_redundant,bad_claim(all 3 legs, and each leg alone),stalemissing,undeclared,unpinned,unheld,downgradedπ‘ N4 β two copies of the resolver, collapsed
The ask ledger carried a private
resolve()while the deny side went througheffective_bash_action. Zero disagreements today and nothing pinning that. Both now route through one_resolve_over(rules, command)+_rules_without(rules, drop).π‘ N5 β the floor's paragraph was prose, not a gate
It claimed "the literal is still raised when one lands, so the number never drifts back into being a cushion". Measured: adding an ask rule and pinning it, literal left at 51, was GREEN under
>=β the cushion regrows one rule at a time until it is exactly the slack the same docstring spends two paragraphs condemning.Now
== 51, with a message naming both directions:>=(old)==(new)Deferred, and honestly labelled
--decryptcontains-d; everyrestartcontainsstart).*terraform*destroy*control is gone, replaced byASK_CONTROLS[0]over a guaranteed-absentSENTINEL, plus a test asserting the sentinel is absent.wrongly_redundant's message still names a scenariobad_claimpreempts. Left as-is.nix store gc,nix-store --gc,nix profile wipe-history,nix-env --delete-generations old,home-manager expire-generations;pkexec/run0bypass the sudo/doas block;FOO=1 sudo-rs whoamiand/usr/bin/sudo-rs whoamiescape the anchoredsudo*β notesudo-rsis the witness this PR picks forsudo*, so that pin is narrower than it looks;rm -vr,rm -Rv,rm -f -R,FOO=1 rm -Rare live spellings the characterization test does not pin; and*home-manager*switch*/*systemctl*daemon-reload*survive a trailing-*strip, so the househome-manager switch --flake . --impureform is not itself pinned.Incoming conflict to expect
The concurrent session's uncommitted WIP in
scripts/claude-hooks/tests/test_guard_core.pyadds agit addblock and states it will remove"*git*add ."fromopencode.jsonc. This PR'sGUARD_BACKSTOPPED_DENIESdeclares that rule, and the deny ledger pins the deny set β so whichever lands second must move that entry, and the test will say so by name. Mytimetests are appended at EOF specifically to avoid a textual clash with their insert at lines 412-498. I read that file read-only and did not modify their WIP.Verification β mechanism stated for every number
nix build .#checks.x86_64-linux.pytestsβ the flake-pinned sandbox gate.buildEnvfrom this flake's nixpkgs,opencode --versionasserted 1.18.4 (an ad-hocnix-shell -p opencodeserves 1.18.11) β the mutation batteries and the N1/N2/N5 matrices.opencode debug agent build --purefrom that same pinned binary, stdout read from a FILE β the 8192-byte pipe truncation attest_opencode_engine.py:175is real and pre-existing; not fixed here, not repeated here.Full mutant battery, re-run from scratch on the final tree (78 rows; baseline 628 passed / 0 failed):
allowdenyβaskBranch reachability: 12 / 12, each failing with its own message. One case had to be retargeted β
deny/downgraded's mutant patched a function the ledger no longer calls after the N4 dedupe, and it went green; that is the same shadowing hazard the harness exists to catch, so it is worth naming.Engine: all 26 pins resolve
askagainst the real 1.18.4 engine with the intended rule LAST β none shadowed. guard_core suite: 1156 passed. #350's new scanners pass against my edits (1817 passed across the two scanners +test_guard_core.py+test_opencode_config.py), so the count it pins intest_guard_core.pyis intact.Both tiers, both green:
origin/main@60e6d9d(built standalone as control)origin/main(I measured the control rather than reusing the briefed 6164/6163 β that figure predates #350, which added ~147 tests.)
π΄ Deploy note
~/.config/opencode/opencode.jsoncis ahome.filecopy resolving into/nix/store. Merging does not activate the new"*sudoedit*"rule, and merging does not deploy thetimefix to~/.claude/hooks/either. Both needhome-manager switch(orscripts/ship.sh) on each host. Merged β deployed.