Recall-failure fixes: live-validate gate in /pr-create + fixture-realism in hook-authoring - #309
Conversation
Two harness-level recall fixes for lessons memory alone failed to fire:
1. /pr-create: new step 4 gates on `git diff --name-only` matching
home/.claude/{hooks,plugins,settings.json}. Surfaces a live-validation
prompt before PR creation — for behavioral guards (hook/lint/policy
that BLOCKS something), CI fixtures can't prove the guard is wired
in and reachable. References the new memory.
2. hook-authoring SKILL.md: "Fixture realism" nudge in the Testing
section. Sample real transcript content before writing regex; don't
hand-write fixtures from a mental model.
Both surfaced by improve-workflow reflection on the three-PR Stop-hook
arc (#303 → #307 → #308) where each round shipped with insufficient
empirical validation. Skill prose stays tight (~3-5 lines each) to
respect the cost-per-invocation rule from PR #306.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Review by Claude Code (claude-review.md). Doc-only PR; no Critical or Important issues. Three Suggestion-level findings below — non-blocking. Note: harness blocked file writes needed for inline-comment JSON, so findings are consolidated here with file:line anchors.
Verdict: APPROVE
Suggestion 1 — home/.claude/skills/hook-authoring/SKILL.md:209 — Dangling reference. The new line says 'See hooks/README.md Gotchas for the empirical case study (regex shipped without backtick support...)' but home/.claude/hooks/README.md Gotchas (line 266+) actually covers jq multiline regex anchoring, no-jq test PATH setup, and Stop hook transcript-writer races — none of them is the backtick case study. The backtick-wrapped-decorator narrative actually lives inline in home/.claude/hooks/enforce-insight-publish.sh:60-68. Either point the reader there (See enforce-insight-publish.sh:60-68 for the empirical case study) or add a short Gotchas entry in hooks/README.md so the reference resolves.
Suggestion 2 — home/.claude/commands/pr-create.md:49 — The home/.claude/plugins/ arm of the path-gate is dead today: that directory doesn't exist in the dotfiles repo (plugins are referenced from settings.json enabledPlugins, not vendored as files). Either drop home/.claude/plugins/ from the list to keep the gate honest, or add a brief inline note that it's a forward-looking hook for if/when plugins get vendored locally.
Suggestion 3 — home/.claude/commands/pr-create.md:49 — The trigger is described in prose ('if git diff --name-only main...HEAD matches ...') rather than as a concrete pattern, which leaves the match semantics to the model and could drift between invocations. A one-liner like 'git diff --name-only main...HEAD | grep -qE ^home/[.]claude/(hooks/|settings[.]json$)' would make the trigger deterministic and align with the harness-level check framing called out in the PR description (and is cheap — grep -q exits early).
- pr-create.md: replace prose trigger with deterministic `grep -qE` one-liner, and drop the dead `home/.claude/plugins/` arm (plugins aren't vendored in this repo; they're referenced via enabledPlugins in settings.json). - hook-authoring SKILL.md: fix dangling reference. The backtick case study lives inline in enforce-insight-publish.sh, not in hooks/README.md Gotchas (which covers jq multiline, PATH-stubbing, and the transcript race — not backticks). Point at the script with its inline comment. Verified the new regex via dry-run: matches `home/.claude/hooks/foo.sh` and `home/.claude/settings.json`, correctly does NOT match `home/.claude/skills/` or `home/.claude/commands/`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Feedback AddressedImplemented
|
There was a problem hiding this comment.
Follow-up review. All three prior Suggestion-level findings (dangling reference, dead plugins arm, prose-only trigger) are correctly addressed. One new Suggestion: in home/.claude/skills/hook-authoring/SKILL.md:209, the new line contains 9 backticks (odd), so at least one inline code span is unmatched. The regex around match(...) renders awkwardly on GitHub because an inner backtick splits it into two spans plus a literal backtick. Easy fixes: wrap the whole regex in double-backticks so the inner backtick becomes a literal, or refer to the call by location (See the match call at enforce-insight-publish.sh:87 and the inline comment above it). APPROVE - one Suggestion, non-blocking. Automated review by Claude Code.
PR #309 round-2 review caught that my dangling-reference fix introduced an odd backtick count (9) on hook-authoring/SKILL.md:209 — the inlined regex contained a literal backtick that closed an inline code span prematurely. Drop the inlined regex from the prose entirely; the file pointer itself is the durable reference, and the script's inline comment has the regex + narrative anyway. Backtick count is now even (6, three pairs). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Feedback Addressed (round 2)Implemented
|
There was a problem hiding this comment.
Review by Claude Code (claude-review.md). Doc-only PR, round 3 review. All four prior Suggestion-level findings (dangling reference, dead plugins arm, prose-only trigger, odd-backtick markdown) are correctly resolved. Verified: enforce-insight-publish.sh exists with the backtick-aware regex comment at lines 61-87 — the SKILL.md pointer resolves cleanly; the grep -qE pattern in pr-create.md correctly matches home/.claude/hooks/* and home/.claude/settings.json (exact), correctly skips settings.local.json, skills/, and commands/; no leftover plugins/ arm; SKILL.md backtick count is balanced. No Critical, Important, or Suggestion-level issues. Verdict: APPROVE.
Summary
Two surgical edits surfaced by
improve-workflowreflection on the recent three-PR Stop-hook enforcement arc (#303 → #307 → #308):home/.claude/commands/pr-create.md— new step 4 (path-gated live-validation prompt). Whengit diff --name-only main...HEADmatcheshome/.claude/hooks/,plugins/, orsettings.json, surface a live-validate question before opening the PR. Five lines of prose.home/.claude/skills/hook-authoring/SKILL.md— fixture-realism nudge in the Testing section. Sample one real~/.claude/projects/*/<id>.jsonlmessage before writing regex fixtures, instead of hand-writing the expected format. Three lines of prose.Why these specifically
The three-PR arc had a recurring failure mode: memory entries existed for "live-validate runtime PRs before merge" but didn't fire when I needed them, because at PR-creation time I was thinking "the regex fix is done" rather than "is the live-validation gap applicable here." Same pattern across rounds 1 and 2:
The diagnosis: memory alone is recall-dependent; the rule needs to fire deterministically when the path-match condition holds. Hence the harness-level check in
/pr-createrather than more prose in CLAUDE.md.Cost discipline
Both edits respect the cost-per-invocation rule shipped in #306:
pr-create.mdgets +5 lines of prose, paid every/pr-createinvocation. The check is path-gated so the AskUserQuestion only fires when relevant — most invocations just read the condition and move on.hook-authoringSKILL.md auto-loads only when the model detects hook-authoring context. +3 lines of prose, bounded cost.Avoided: the broader
/workenforcement-gap classifier (event #4029) which had similar motivation but a fuzzier trigger and would have taxed every/workinvocation. Kept that one in memory + event bus.Companion memory
feedback_behavioral_guard_acceptance.md(in personal memory, not in this repo): for hook/lint/policy PRs, acceptance = deliberate violation in a fresh session. Replay against a transcript without a fresh violation proves nothing. Referenced from the/pr-createstep.Test plan
make checkpasses (29/29; doc-only changes, no shell or test impact)git diff --name-only+ AskUserQuestion — deterministic.🤖 Generated with Claude Code