feat(hooks+eval): claim-scan detector fix + session-bound groundedness eval (Q-E1…Q-E4)

[artyhoo]

Implements the maintainer's decisions on the instruction-compliance-empirical R-phase (#97, merged). Umbrella authorized 2026-05-21 («сделай всё как лучше»). Atomic commits, one concern each.

Decisions realised

Decision	Verdict	Commit
Q-E1 which gate	DEFER clean A/B + FIX detector	(whole PR)
Q-E4 detector fix	recall ≤2 intervening tokens + precision strip citations/quotes/links	fix(hooks)
Q-E2 fund longitudinal eval	scorer → committed session-bound tool	feat(eval)
Q-E3 §13.34 trigger	≥+15pp grounded-lift gate	docs

Q-E4 — detector fix (.claude/hooks/end-of-turn-reminder.sh)

The R-phase measured the live detector: recall ~0.43 on natural numeric phrasings (adjacency-only regex missed "6 discipline rules", "11 distinct principles", and the pilot's one organic failure case); precision ~0.20–0.25 (over-fired on numbers/paths in fenced code, blockquotes, link targets).

• Numeric regex now allows ≤2 intervening tokens.
• Scan runs over a cleaned copy (fenced code + blockquotes + markdown link targets stripped); inline code kept so genuine file:line citations still fire.
• tests/hooks/eot-claim-scan.test.sh → 9 sub-tests (recall intervening-token case + 2 precision cases + shape-robust mutation). All green.
• Validated on 340 real transcripts: recall +~37% claim-turns detected, precision −~105 false-positive claim-turns. Noun-list extension deliberately left out of scope (separate axis).

Q-E2 — session-bound eval (tests/eval/)

claim-groundedness-scorer.py (deterministic, no-LLM) + README + committed baseline-2026-05-21. NOT a CI gate — needs the operator's private transcripts + no-paid-llm-in-ci; operator-run like template-audit/audit-self. Regexes + cleaning kept identical to the hook so the eval measures the same surface. SSOT #53 records the build-vs-reuse verdict (BUILD thin scorer; promptfoo/Inspect/METR REFERENCE — they host the grading half but not the no-paid-LLM generation half).

Q-E3 — §13.34 promotion gate

The claim-scan stays an interim until the eval shows ≥+15pp grounded-lift over baseline (≥50 fired instances); else candidate for removal or the H10-pair target. Clean A/B deferred to longitudinal accrual (≈no claim-scan-active data yet). Decisions recorded in patch §8.5 (mirrors #95 §10.5).

Recursive note

The detector fix was validated by re-running the eval the same R-phase designed — the eval is the test the fix had to pass («documents lie; tests don't»).

§1.7: hook fix + eval tool extend existing mechanisms / add session-bound tooling — no new .claude/rules/ rule introduced. Principle suite 81/81 green; markdownlint clean; capability gate not tripped (tool under tests/, not packages/).

---

§1.7 Forward-check applied

• The claim-detection regexes + cleaning are now a shared spec across two channels — the live hook (.claude/hooks/end-of-turn-reminder.sh:76) and the eval scorer (tests/eval/claim-groundedness-scorer.py:31). Forward obligation (dual-implementation-discipline §7): any future regex change MUST update both; the hook header note at .claude/hooks/end-of-turn-reminder.sh:67 states this so the eval keeps measuring the same surface.
• §13.34 now carries an empirical gate (docs/meta-factory/open-questions.md:420): every future «promote the interim» decision must first run tests/eval/ against the committed baseline — the promotion is no longer belief-based.

§1.7 Backward-check applied

• The merged research patch (docs(research-patches): instruction-compliance vs structural-forcing empirical research #97) §6.4 documented the recall blind spot as an open finding; this PR's appended §8.5 (docs/meta-factory/research-patches/2026-05-21-instruction-compliance-empirical.md:186) records the maintainer decisions that close it — no upstream doc now calls the gap «unaddressed».
• The companion test now asserts the exact recall case the patch identified as missed (tests/hooks/eot-claim-scan.test.sh:99), back-linking the fix to its empirical origin. feat(hooks): factual-claim scan in end-of-turn Stop hook (autonomous-self-audit §11.1 interim) #96's Q-D recursive backstop (a test catches silent hook breakage) still holds — the mutation sub-test was updated to the new regex shape, not dropped.