Skip to content

feat(hooks): PostToolUse[Bash] nudge toward the native Grep/Glob tools - #297

Merged
ZacxDev merged 1 commit into
mainfrom
feat/search-tool-nudge-hook
Aug 3, 2026
Merged

feat(hooks): PostToolUse[Bash] nudge toward the native Grep/Glob tools#297
ZacxDev merged 1 commit into
mainfrom
feat/search-tool-nudge-hook

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Why

RULES.md's "Tool Optimization" section already says "Grep over bash grep, Glob over find". Measured over a 30-day activity-telemetry window, it does not work:

host Bash calls Grep Glob
workbench 31,355 33 17
laptop 6,164 0 0

Bash is 71% of all tool calls; compliance is 50 calls against 37,519. Per RULES.md "Deterministic Over Prose", the replacement has to be structural — so nudge at the moment the search-shaped command runs.

What

scripts/claude-hooks/search-tool-nudge.py — a PostToolUse[Bash] hook that detects a tree search and injects additionalContext pointing at Grep/Glob.

🔴 It is a NUDGE, not a block. It never emits a deny decision and always exits 0 (asserted by a test: io never denies). Same shape as shell-env-nudge.py / audit-pr-nudge.py; bash-guard.py remains the only hook that denies.

Fires on: grep -r/-R/--recursive (incl. clusters like -rn) · bare rg/ag/ack · find <path> -name/-iname/-path/-regex · ls -R · find … | xargs grep · find … -exec grep. Deduped once per KIND per session (kinds: content→Grep, files→Glob), so a session sees at most two of these ever.

Deliberately silent on (false positives are the expensive failure mode on a per-Bash-call hook):

  • grep TODO README.md — a single named file
  • git log | grep push — a pipeline filter (called out in the task as a known over-match hazard), plus ps aux | grep, systemctl … | grep, kubectl … | grep -v, journalctl | rg
  • find … -exec rm/chmod, find … -delete, find /var/log -mtime +30 (no name predicate)
  • plain ls -la
  • search-shaped text inside a quoted string, incl. ssh host 'grep -r foo /etc'
  • heredoc bodiesssh host 'bash -s' <<'EOF' … EOF runs on a REMOTE host that Grep/Glob cannot reach. Found only by running the detector over the real transcript corpus.

Design decision: new file, not an extension of shell-env-nudge.py

New file. Trade-off, stated plainly: it adds a third per-Bash-call subprocess — measured at 28 ms mean for a non-firing call. Extending shell-env-nudge.py would avoid that but give one hook two unrelated responsibilities, two dedupe namespaces in one cache, and a name that lies about half of what it does. Two PostToolUse[Bash] nudge subprocesses already exist, so this follows an established one-hook-one-concern pattern rather than introducing a new one. If the per-call cost ever matters, the right consolidation is all three nudges behind one dispatcher — not folding two of them together now.

Also in this PR: a silent-skip fix in run-tests.sh

⚠️ This PR edits scripts/run-tests.sh (the hand-rolled HOOK_TESTS list, ~line 400) — flagged for merge-order checking against the sibling PRs touching guard_core.py and session-analysis/insights.py.

Two changes there:

  1. Added scripts/claude-hooks/tests/test_search_tool_nudge.py to HOOK_TESTS.
  2. The loop's [ -f "$HOOK_TEST" ] || continue was a silent skip — the exact feat(opencode): home-manager-managed opencode config — generated AGENTS.md, env plugin, permissions, 3 subagents #276 shape (a file added to a target list, silently rejected, gate green while 913 tests never ran). A missing entry is now a loud FAIL.

Verification

All verification was done by invoking the committed source directly with PostToolUse JSON payloads. Nothing was deployed; home-manager switch was NOT run. These are claims about the source in this commit, not about a deployed artifact.

Control pair (requirement 4)

The reassuring answer here is a zero, so the zero is never reported alone:

  • Harness negative controlcheck() is fed a deliberate mismatch at the top of the file and the run aborts with exit 2 if that does not register as a failure. Until it has been watched go red, a green from this file is a fact about the harness.
  • Benign-set positive control1 on the positive control (grep -r foo src/through the samefire_count() path), 0 false positives on the 23-case benign set. A counter wired to nothing also reports 0; this one is shown to move.

Runner-entry controls (requirement 3 — verified EXECUTED, not assumed)

  • Positive: flake check output contains PASS scripts/claude-hooks/tests/test_search_tool_nudge.py (script).
  • Negative: with the test file moved aside, the runner printed run-tests: ERROR — hook test 'scripts/claude-hooks/tests/test_search_tool_nudge.py' does not exist and FAIL … (missing). Before this PR's fix that case was silent.

Red/green matrix (requirement 5)

ref result
origin/main @ 73f5ec0 (git archive of scripts/claude-hooks/, test dropped in) 🔴 REDFileNotFoundError: …/search-tool-nudge.py, rc=1
HEAD of this branch 🟢 GREENall search-tool-nudge tests passed (58 checks: 22 must-fire, 23 benign, 1 harness negative control, 1 benign-counter positive control)

Absence-at-base is a weak red, so it is backed by a mutation sweep (13 mutants + an identity control). Every mutant went RED naming the specific check, and the identity control passed — proving the sweep harness can distinguish. Two examples: M2 grep fires even when not recursiveBENIGN grep single file: got ['content'] want []; M11 newlines not turned into separatorsFIRE search on a later LINE: got [] want ['content'].

M11 was a real bug the sweep found, not a synthetic one: shlex treats \n as ordinary whitespace, so cd /repo\ngrep -r foo . lexed as a single cd command and the search was invisible. Fixed with a quote-aware newline→separator pass. Two mutants (M11b) are reported as EQUIVALENT — the pre-pass's quote/backslash tracking survives mutation because shlex re-parses quotes itself. That is stated in the code's docstring rather than papered over with a contrived test.

Real-corpus false-positive audit

The detector was run over 27,315 real Bash commands from the local Claude transcripts: 2,589 fire (9.5%). Two random samples of 30 firing commands (one general, one restricted to multi-line commands) were read individually — all 60 were genuine local tree searches; no false positive found. This audit is what surfaced the heredoc class, which was then pinned as a test.

Suite counts (requirement 8 — counted, not exit-coded)

nix build .#checks.x86_64-linux.pytestsTOTAL collected=4373 passed=4372 skipped=1 failed=0 (floor: 2850), RESULT: PASS. Hook tests are pass/fail only and by design not part of TOTAL, so that number is unchanged; what moved is HOOK_TESTS 4 → 5 entries and this file's own 58 counted checks (with a MIN_CHECKS = 55 floor so a silently-shrunk suite fails).

🔴 Per-host action required after merge

~/.claude/settings.json is per-host and NOT nix-managed. register-nudge-hook.py was extended to register this hook (append-only, idempotent, atomic write — covered by two new assertions in test_register_nudge_hook.py). It must be re-run on BOTH hosts after the home-manager switch:

python3 ~/workspace/devrc/scripts/claude-hooks/register-nudge-hook.py

Sequence is merge → pull → ship.sh (switch) → run the registrant per host. Until the registrant runs, the script is deployed but never invoked.

🤖 Generated with Claude Code

RULES.md's "Tool Optimization" section says "Grep over bash grep, Glob over find".
Measured over a 30-day telemetry window that prose rule is not working: Bash is 71%
of all Claude tool calls (workbench 31,355 / laptop 6,164) against 50 Grep+Glob calls
total — and ZERO on the laptop. Per RULES.md "Deterministic Over Prose", the
replacement has to be structural, so nudge at the moment the search runs.

search-tool-nudge.py fires PostToolUse on Bash calls that are a tree search
(`grep -r`, bare `rg`/`ag`/`ack`, `find <path> -name`, `ls -R`, `find | xargs grep`,
`find -exec grep`) and injects additionalContext pointing at Grep/Glob. It is a
NUDGE: it never denies and always exits 0, matching shell-env-nudge/audit-pr-nudge.
Deduped to once per KIND (content / files) per session, so a session sees at most two.

Conservative by construction, because false positives on a per-Bash-call hook train
the operator to ignore it. Silent on: a non-recursive grep (single file OR a pipeline
filter such as `git log | grep push`), `rg` fed by a pipe, `find -exec rm/chmod`,
`find -delete`, `find` with no name/path predicate, plain `ls`, search-shaped text
inside a quoted string, and heredoc bodies (`ssh host 'bash -s' <<'EOF' … EOF` runs
on a REMOTE host that Grep/Glob cannot reach).

Also fixes the silent-skip in run-tests.sh's HOOK_TESTS loop: a missing entry was
`|| continue`, the exact #276 shape GUARD 5 exists to prevent. It now fails loudly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ZacxDev
ZacxDev merged commit 289ffa6 into main Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant