feat(hooks): PostToolUse[Bash] nudge toward the native Grep/Glob tools - #297
Merged
Conversation
RULES.md's "Tool Optimization" section says "Grep over bash grep, Glob over find". Measured over a 30-day telemetry window that prose rule is not working: Bash is 71% of all Claude tool calls (workbench 31,355 / laptop 6,164) against 50 Grep+Glob calls total — and ZERO on the laptop. Per RULES.md "Deterministic Over Prose", the replacement has to be structural, so nudge at the moment the search runs. search-tool-nudge.py fires PostToolUse on Bash calls that are a tree search (`grep -r`, bare `rg`/`ag`/`ack`, `find <path> -name`, `ls -R`, `find | xargs grep`, `find -exec grep`) and injects additionalContext pointing at Grep/Glob. It is a NUDGE: it never denies and always exits 0, matching shell-env-nudge/audit-pr-nudge. Deduped to once per KIND (content / files) per session, so a session sees at most two. Conservative by construction, because false positives on a per-Bash-call hook train the operator to ignore it. Silent on: a non-recursive grep (single file OR a pipeline filter such as `git log | grep push`), `rg` fed by a pipe, `find -exec rm/chmod`, `find -delete`, `find` with no name/path predicate, plain `ls`, search-shaped text inside a quoted string, and heredoc bodies (`ssh host 'bash -s' <<'EOF' … EOF` runs on a REMOTE host that Grep/Glob cannot reach). Also fixes the silent-skip in run-tests.sh's HOOK_TESTS loop: a missing entry was `|| continue`, the exact #276 shape GUARD 5 exists to prevent. It now fails loudly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
RULES.md's "Tool Optimization" section already says "Grep over bash grep, Glob over find". Measured over a 30-day activity-telemetry window, it does not work:
Bash is 71% of all tool calls; compliance is 50 calls against 37,519. Per RULES.md "Deterministic Over Prose", the replacement has to be structural — so nudge at the moment the search-shaped command runs.
What
scripts/claude-hooks/search-tool-nudge.py— a PostToolUse[Bash] hook that detects a tree search and injectsadditionalContextpointing at Grep/Glob.🔴 It is a NUDGE, not a block. It never emits a deny decision and always exits 0 (asserted by a test:
io never denies). Same shape asshell-env-nudge.py/audit-pr-nudge.py;bash-guard.pyremains the only hook that denies.Fires on:
grep -r/-R/--recursive(incl. clusters like-rn) · barerg/ag/ack·find <path> -name/-iname/-path/-regex·ls -R·find … | xargs grep·find … -exec grep. Deduped once per KIND per session (kinds: content→Grep, files→Glob), so a session sees at most two of these ever.Deliberately silent on (false positives are the expensive failure mode on a per-Bash-call hook):
grep TODO README.md— a single named filegit log | grep push— a pipeline filter (called out in the task as a known over-match hazard), plusps aux | grep,systemctl … | grep,kubectl … | grep -v,journalctl | rgfind … -exec rm/chmod,find … -delete,find /var/log -mtime +30(no name predicate)ls -lassh host 'grep -r foo /etc'ssh host 'bash -s' <<'EOF' … EOFruns on a REMOTE host that Grep/Glob cannot reach. Found only by running the detector over the real transcript corpus.Design decision: new file, not an extension of
shell-env-nudge.pyNew file. Trade-off, stated plainly: it adds a third per-Bash-call subprocess — measured at 28 ms mean for a non-firing call. Extending
shell-env-nudge.pywould avoid that but give one hook two unrelated responsibilities, two dedupe namespaces in one cache, and a name that lies about half of what it does. Two PostToolUse[Bash] nudge subprocesses already exist, so this follows an established one-hook-one-concern pattern rather than introducing a new one. If the per-call cost ever matters, the right consolidation is all three nudges behind one dispatcher — not folding two of them together now.Also in this PR: a silent-skip fix in
run-tests.shscripts/run-tests.sh(the hand-rolledHOOK_TESTSlist, ~line 400) — flagged for merge-order checking against the sibling PRs touchingguard_core.pyandsession-analysis/insights.py.Two changes there:
scripts/claude-hooks/tests/test_search_tool_nudge.pytoHOOK_TESTS.[ -f "$HOOK_TEST" ] || continuewas a silent skip — the exact feat(opencode): home-manager-managed opencode config — generated AGENTS.md, env plugin, permissions, 3 subagents #276 shape (a file added to a target list, silently rejected, gate green while 913 tests never ran). A missing entry is now a loudFAIL.Verification
All verification was done by invoking the committed source directly with PostToolUse JSON payloads. Nothing was deployed;
home-manager switchwas NOT run. These are claims about the source in this commit, not about a deployed artifact.Control pair (requirement 4)
The reassuring answer here is a zero, so the zero is never reported alone:
check()is fed a deliberate mismatch at the top of the file and the run aborts with exit 2 if that does not register as a failure. Until it has been watched go red, a green from this file is a fact about the harness.1 on the positive control (grep -r foo src/through the samefire_count()path), 0 false positives on the 23-case benign set.A counter wired to nothing also reports 0; this one is shown to move.Runner-entry controls (requirement 3 — verified EXECUTED, not assumed)
PASS scripts/claude-hooks/tests/test_search_tool_nudge.py (script).run-tests: ERROR — hook test 'scripts/claude-hooks/tests/test_search_tool_nudge.py' does not existandFAIL … (missing). Before this PR's fix that case was silent.Red/green matrix (requirement 5)
origin/main@73f5ec0(git archiveofscripts/claude-hooks/, test dropped in)FileNotFoundError: …/search-tool-nudge.py, rc=1all search-tool-nudge tests passed (58 checks: 22 must-fire, 23 benign, 1 harness negative control, 1 benign-counter positive control)Absence-at-base is a weak red, so it is backed by a mutation sweep (13 mutants + an identity control). Every mutant went RED naming the specific check, and the identity control passed — proving the sweep harness can distinguish. Two examples:
M2 grep fires even when not recursive→BENIGN grep single file: got ['content'] want [];M11 newlines not turned into separators→FIRE search on a later LINE: got [] want ['content'].M11 was a real bug the sweep found, not a synthetic one: shlex treats
\nas ordinary whitespace, socd /repo\ngrep -r foo .lexed as a singlecdcommand and the search was invisible. Fixed with a quote-aware newline→separator pass. Two mutants (M11b) are reported as EQUIVALENT — the pre-pass's quote/backslash tracking survives mutation because shlex re-parses quotes itself. That is stated in the code's docstring rather than papered over with a contrived test.Real-corpus false-positive audit
The detector was run over 27,315 real Bash commands from the local Claude transcripts: 2,589 fire (9.5%). Two random samples of 30 firing commands (one general, one restricted to multi-line commands) were read individually — all 60 were genuine local tree searches; no false positive found. This audit is what surfaced the heredoc class, which was then pinned as a test.
Suite counts (requirement 8 — counted, not exit-coded)
nix build .#checks.x86_64-linux.pytests→TOTAL collected=4373 passed=4372 skipped=1 failed=0 (floor: 2850),RESULT: PASS. Hook tests are pass/fail only and by design not part ofTOTAL, so that number is unchanged; what moved isHOOK_TESTS4 → 5 entries and this file's own 58 counted checks (with aMIN_CHECKS = 55floor so a silently-shrunk suite fails).🔴 Per-host action required after merge
~/.claude/settings.jsonis per-host and NOT nix-managed.register-nudge-hook.pywas extended to register this hook (append-only, idempotent, atomic write — covered by two new assertions intest_register_nudge_hook.py). It must be re-run on BOTH hosts after thehome-manager switch:Sequence is merge → pull →
ship.sh(switch) → run the registrant per host. Until the registrant runs, the script is deployed but never invoked.🤖 Generated with Claude Code