Skip to content

test(hooks): paired-negative test for end-of-turn-reminder Stop hook (M.4.5) - #196

Merged
artyhoo merged 1 commit into
stagingfrom
test/m4-end-of-turn-reminder
May 24, 2026
Merged

test(hooks): paired-negative test for end-of-turn-reminder Stop hook (M.4.5)#196
artyhoo merged 1 commit into
stagingfrom
test/m4-end-of-turn-reminder

Conversation

@artyhoo

@artyhoo artyhoo commented May 24, 2026

Copy link
Copy Markdown
Owner

Summary

Track M.4.5 — paired-negative coverage for .claude/hooks/end-of-turn-reminder.sh (Stop hook), closing one of the 6 untested-hook gaps surfaced in the post-PR #183 channel-earliness audit. Per umbrella kickoff .claude/orchestrator-prompts/m4-bash-hook-tests/kickoff.md §1 row 5.

Single new test file: packages/core/hooks/end-of-turn-reminder.test.ts (314 LOC, 12 tests). Zero production code change.

Test shape

  • ❌ trigger turns must emit JSON {decision:"block", reason, systemMessage}:
    Branch A (long markdown), Branch B (bare question), Branch C (long+question),
    Branch D (file:line claim), AskUserQuestion-only tool_use.
  • ✅ skip conditions exit 0 silent: stop_hook_active, missing transcript, no
    assistant turn, short non-claim turn.
  • Boundary (precision): file:line claim inside a fenced code block does NOT
    count as a claim (hook:71-74 cleaner-fix guard).
  • Regression guard (B2): AskUserQuestion-only turn after a prior ## 🟢 recap
    must FIRE, not be eaten by grep -qF "" matching everything (hook:128-131 fix).

Per T-M4-B the test asserts payload SHAPE — decision, reason (non-trivial string), systemMessage matching 🎯 <anchor> — not just exit code. Exit-only would have missed both the #81 broken-systemMessage-delivery bug and the round-1 AUQ false-suppress regression that prior cold-review caught.

Mutation-sanity (T19)

Verified the test has teeth: mutating production hook line 256 decision: "block""allow" makes 6 trigger tests fail; the 6 silent-exit tests correctly stay green. Hook restored before commit; diff -q against backup = 0.

Test plan

  • npx vitest run hooks/end-of-turn-reminder.test.ts12/12 green at packages/core/hooks/end-of-turn-reminder.test.ts:1
  • tsc -p packages/core --noEmit — clean
  • Mutation-sanity: 6 trigger tests fail on decision: "block"→"allow"; restored
  • pnpm test full suite — 825 passed / 4 skipped / 1 failed; the 1 failure is
    pre-existing on origin/staging in hooks/pre-push.test.ts:54-61 (asserts
    audit-ai-docs.test.sh / hook-stub-completeness.test.sh literal paths, but
    Wave 10.4 / 10.6 migrated those to .ts — see packages/core/hooks/pre-push.ts
    inline comment «audit-ai-docs.test.ts (Wave 10.4): run via vitest (replaces
    audit-ai-docs.test.sh)»). CI on origin/staging is green (audit-self workflow
    filters its vitest scope), confirming this failure is outside the gating set.
    Surfaced as observation per kickoff §2 + T5 (no fix bundled into M.4.5).

§1.7 Forward-check applied

  • no-paid-llm-in-ci.md §1 — test uses
    deterministic vitest + spawnSync of the bash hook with on-disk JSONL fixture
    (packages/core/hooks/end-of-turn-reminder.test.ts:65-72); zero API-billed call.
  • build-first-reuse-default.md §1
    verdict REFERENCE: test file leading docstring + spawnSync pattern at
    packages/core/hooks/end-of-turn-reminder.test.ts:30-48 mirrors precedent
    packages/core/hooks/check-hook-marker.test.ts:50-64. No new framework, no new
    dep, no SSOT row needed.
  • paired-negative principle 02
    ❌ trigger tests at packages/core/hooks/end-of-turn-reminder.test.ts:118-218
    paired with ✅ silent-exit tests at :222-264 and a precision boundary at :271-285.
  • ai-laziness-traps.md §2 — active T-traps:
    T3 (file:line cites against .claude/hooks/end-of-turn-reminder.sh:7-15,128-131,249-259
    throughout assertions), T5 (pre-push.test.ts pre-existing failure surfaced
    as observation only — not fixed in this PR), T11 (REFERENCEs precedent at
    packages/core/hooks/check-hook-marker.test.ts:50-64 in the docstring), T13
    (asserts payload SHAPE per T-M4-B at :127-131,143-148,159-167,180-187,199-204,
    not exit code only), T15 (test files are leaves — meta-tests-of-tests would
    be infinite regress; noted here pre-emptively), T19 (own mutation-sanity ran
    before push: 6/6 trigger tests failed on decision:"block"→"allow" mutation,
    hook restored, diff -q = 0).
  • Domain-specific (kickoff §7): T-M4-A (TS + spawnSync, no bats), T-M4-B
    (payload-shape assertions on decision/reason/systemMessage, not exit-only),
    T-M4-C (Stop-hook contract specifically — reason → MODEL, systemMessage
    USER per memory project_eot_hook_redesign_approved 2026-05-22; verified via
    hook source .claude/hooks/end-of-turn-reminder.sh:249-254 cited contract).

§1.7 Backward-check applied

  • Scope-additive: exactly 1 new file (packages/core/hooks/end-of-turn-reminder.test.ts:1-314);
    zero file deletion; zero modification of .claude/hooks/end-of-turn-reminder.sh
    verified via diff -q /tmp/eot-hook-backup.sh .claude/hooks/end-of-turn-reminder.sh
    returning identical after mutation-sanity restore.
  • No existing artefact silently superseded. The 3 already-tested-hook precedents
    (check-hook-marker.test.ts, check-kickoff-traps.test.ts,
    inject-matching-rule.test.ts) remain unchanged; this PR adds a 4th sibling, not
    a replacement.
  • No dual-implementation markers added to legacy hooks in this PR
    (per kickoff §6 — that is a maintainer-batched separate item per
    dual-implementation-discipline.md §9
    forward-going convention).
  • Pre-existing CI-failure observation (pre-push.test.ts:54-61) surfaced in
    test-plan section above; not fixed here (T5 — no implementation bundled into
    test-add PR; maintainer decides scope).

Unblocks

Tracks 4/6 of M.4 once siblings M.4.1, M.4.2, M.4.3, M.4.4, M.4.6 also merge → unblocks Track M.5a (top-3 edit-time hooks: manifest render-drift / research-patch §1.7-substance / actionlint per-yml) per kickoff §3 downstream gate.

…ook (M.4.5)

Track M.4.5 — paired-negative coverage for `.claude/hooks/end-of-turn-reminder.sh`
per kickoff `.claude/orchestrator-prompts/m4-bash-hook-tests/kickoff.md` §1 row 5.
Closes one of the 6 untested-hook gaps surfaced in the post-PR-#183 audit.

12 tests covering:
- ❌ trigger turns must emit JSON `{decision: "block", reason, systemMessage}`:
  Branch A (long markdown), Branch B (bare question), Branch C (long + question),
  Branch D (file:line claim), AskUserQuestion tool_use path.
- ✅ skip conditions exit 0 silent: stop_hook_active, missing transcript,
  no assistant turn, short non-claim turn.
- Boundary: claim inside code fence is NOT a claim (precision guard, hook:71-74).
- Regression guard: B2 idle-suppress fix (hook:128-131) — AUQ-only turn after a
  prior "## 🟢" recap must FIRE, not be eaten by `grep -qF ""` matching anything.

Per T-M4-B the test asserts payload shape (decision/reason/systemMessage), not
just exit code — exit-only would have missed the #81 broken-systemMessage and
the round-1 AUQ false-suppress regression that prior cold-review caught.

Mutation-sanity verified: `decision: "block"` → `"allow"` in the production
hook makes all 6 trigger tests fail; hook restored before commit (git diff = 0).

REFERENCEs `check-hook-marker.test.ts:50-64` spawnSync+JSONL-fixture pattern,
no new test framework or dep. No production code changed (T5).

Prior-art: skipped — test additions for existing capability (see kickoff §3).
@artyhoo
artyhoo merged commit 9b37792 into staging May 24, 2026
22 checks passed
artyhoo added a commit that referenced this pull request May 25, 2026
… detection) (#220)

Run `bash .claude/skills/meta-orchestrator/helpers/plan-currency-check.sh`
(L2 Stage 3 detection shipped in #217) → 88 UNTRACKED-N entries surfaced
between the 2026-05-22 reconciliation and origin/staging tip (#217). Map
each to an existing §0 / Track row by adding the PR number to its evidence
cell, or to a new §0 row for two umbrellas that landed in full since the
prior snapshot. Re-run helper → 0 UNTRACKED remaining.

Key changes:

- Snapshot date 2026-05-22 → 2026-05-25 (header + §0).
- N8 row: A-phase 🔲 → 🟡 — C1 SSOT-existence (#170), C2 kickoff
  T-enumeration floor (#174), C3 principle 13 §1.7 substance (#178),
  C4 delivery-channel marker (#177), activation #180. C5 + cost-levers
  remain gated on §5.3 utilisation trigger.
- Track M.1 / M.4 → DONE: M.1 codified T20 via #212 (with NB note —
  recommendation-laziness took the T20 slot, mutation-equivalence
  T-bump 20→21 still pending); M.4 6 paired-negative bash-hook tests
  shipped #195/#196/#197/#198/#199/#200.
- Two new §0 rows: Meta-orchestrator skill (Track P) — BUILD #186 +
  audit rounds #192/#193/#194/#201/#202 + UX refactor #203/#204/#205
  + planner-completeness #213/#214/#217 + §1.7 PR-body mandate #216;
  Recommendation-laziness discipline — R-phase #206/#207, benchmark
  #210, I-phase Sub-waves A/C/D #211/#212/#215.
- N7 row: + dogfood research-patch #135 / §4 demotion #166 / live-trial
  verified #171. N4b row: + design #136 / record #118.
- Infra paragraph: PR refs for I.1 follow-ups (#121/#123/#124/#125/
  #128/#130/#131/#143/#145/#146/#147/#148/#149/#172/#187), I.2
  (#139/#142/#154/#175), I.3 DN-4 (#126/#132/#133/#138/#140/#152/
  #159/#161/#162/#167).
- Track 2.3 (channel-earliness audit) → DONE 2026-05-23 (#181); removed
  from "What actually remains".
- Footer subsection: standalone work (#191 satellite-arch / #189
  guard-liveness / #173 storm-readiness / #176 §10 port / #182 cleanup),
  Wave 10 follow-ups (#110/#112/#113), plan-revision history (#108/
  #109/#153/#155/#157/#160/#164/#165/#168/#179/#185).

Verification:
- `bash .claude/skills/meta-orchestrator/helpers/plan-currency-check.sh
  | grep -c '^UNTRACKED'` → 0 (down from 88).
- `npx markdownlint-cli docs/meta-factory/wave-sequencing-plan.md`
  → no violations.
- `npx vitest run packages/core/skills/plan-currency-check.test.ts`
  → 14/14 passed.

Prior-art: skipped — chore, doc reconciliation only, no new capability
or rule introduced.
artyhoo pushed a commit that referenced this pull request Jul 3, 2026
Resolves SSOT conflict: staging's #862 took IDs #189-#195 (D1 diagnostics +
#861 backward-sweep-auditor); this branch's capability-reuse entry renumbered
#189 -> #196 with all in-file references updated (rule, agent, research-patch).
install.sh auto-merged (both SHIPPED_DOCS additions kept). Baselines regenerated
for the current agent content. principle 09/11/21 + byte-identical + format:check
re-verified green post-merge.
artyhoo added a commit that referenced this pull request Jul 3, 2026
…capability-reuse-auditor (#863)

Ships **source-before-shape** — an edit-time discipline catching two recurring AI-laziness failures at authoring time, closing a recursive-self-application gap in the project's own operating rules.

Origin (2026-07-02, operator-confirmed cross-session recurrence = promotion trigger):
1. BFR-reinvention — PR #858 shipped a night-mode SKILL.md re-describing the loop SSOT #64 owns; its trailer said "ADAPT #64" while the body re-described it, and it passed principle 11 F1 (which checks trailer presence, not reuse substance). #consult-as-trailer-not-input.
2. scope-from-memory — a launch prompt scoped from recall, not the spec (D1 → B). #claim-from-memory-not-source.

Mechanism (judgment → injection, not a gate):
- Layer A: .claude/rules/source-before-shape.md carries globs/inject markers → the existing inject-matching-rule.sh surfaces the reminder at edit-time (.claude/skills/**, agents/**, .claude/orchestrator-prompts/**). REUSE, zero new engine. Honestly disclosed once-per-session limitation (best-effort first-touch nudge).
- Layer B: agents/capability-reuse-auditor.md — AI-agnostic overlap + trailer↔body auditor (no-paid-llm), doing the semantic pass F1 cannot.

SSOT #196 (ADAPT, records the 6-item BFR-consult); principle 09 REQUIRED_HEADER_DOCS +2; install.sh SHIPPED_DOCS +1 + regenerated byte-identical baselines; AGENTS.md rule-index +1; self-reflection research-patch.

Independently reviewed (3-lens Workflow: 2 approve + 1 adversarial revise applied). Verified: principles green, byte-identical 8/8, injection dogfooded live. Candidate T-trap #consult-as-trailer-not-input surfaced-not-applied to ai-laziness-traps.md (dedicated rule home instead).

§1.7: Forward+backward in source-before-shape.md §6 — complies with no-paid-llm-in-ci.md §1, build-first-reuse-default.md §4, phase-research-coverage.md §1.11, doc-authority-hierarchy.md §2-§3; registered at packages/core/principles/09-doc-authority-hierarchy.ts:54; self-applies (SSOT #196 consult drove the shape, not memory).

Prior-art: prior-art-evaluations.md#196 (source-before-shape mechanism — ADAPT: REUSE inject-matching-rule.sh channel + BUILD the AI-agnostic capability-reuse-auditor; dedup-first ADOPT-VOCABULARY; code-clone tools REJECT on T16 problem-class miss; Superpowers writing-skills REFERENCE).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant