test: paired-negative tests for inject-session-bootstrap hook (M.4.3) by artyhoo · Pull Request #199 · artyhoo/getff

artyhoo · 2026-05-24T11:34:07Z

Summary

Adds packages/core/hooks/inject-session-bootstrap.test.ts — paired-negative vitest tests for the UserPromptSubmit bootstrap-injection hook (Track M.4.3, .claude/hooks/inject-session-bootstrap.sh).

What's tested

Actual hook contract (from hook source, lines 1-14):

UserPromptSubmit hook that always emits a static heredoc to stdout (no session cache, no skip conditions, stdin ignored).
CC harness injects plain stdout directly into prompt context (line 3) — NOT JSON hookSpecificOutput (that's PostToolUse).
Digest bounded by sentinel tags: [session-bootstrap digest — auto-injected at prompt submit] (line 7) / [/session-bootstrap digest] (line 13).

9 tests:

❌ PAIRED-NEGATIVE: stdout MUST NOT be empty (core injection contract, hook lines 6-14)
✅ stdout contains opening sentinel tag (hook line 7)
✅ stdout contains closing sentinel tag (hook line 13)
✅ boundary: opening tag appears before closing tag (both present, in order)
✅ stdout contains goal anchor phrase (hook line 8)
✅ exit code is 0
✅ repeat invocation with same session_id still emits (no false cache — hook has no caching mechanism)
✅ repeat invocations produce identical output (static heredoc)
✅ PAYLOAD SHAPE: output is plain text NOT JSON (UserPromptSubmit ≠ PostToolUse protocol contract)

Mutation-sanity

Verified via /tmp hook copy: mutant (replaced heredoc with true → emits nothing) fails the PAIRED-NEGATIVE "output MUST NOT be empty" assertion. Real hook passes all 9 tests. Production hook NEVER modified — git diff .claude/hooks/inject-session-bootstrap.sh returns empty (verified via git diff HEAD~1 --name-only showing exactly 1 file).

§1.7 Forward-check applied

.claude/rules/no-paid-llm-in-ci.md §1 — vitest + spawnSync, deterministic, zero API/LLM calls. Evidence: packages/core/hooks/inject-session-bootstrap.test.ts:26-76 (entire test body — no external calls).
.claude/rules/build-first-reuse-default.md §1 — REFERENCE: packages/core/hooks/check-hook-marker.test.ts:19-64 pattern (vitest + spawnSync + mkdtempSync harness, reused verbatim).
packages/core/principles/02-paired-negative-test.test.ts — ≥1 ❌ (PAIRED-NEGATIVE at line 56) + ≥1 ✅ (7 positive assertions at lines 62-113) + boundary at packages/core/hooks/inject-session-bootstrap.test.ts:70-78.

§1.7 Backward-check applied

Scope-additive: 1 new test file; zero deletions; zero production change. No existing artifact modified or superseded.

Observations (NOT fixed in this PR per T5)

Pre-push warn-only trigger — the pre-push hook flagged this commit as a "capability commit" (new file ≥80 LOC under packages/). Per CLAUDE.md: "test additions for existing capabilities … NOT capability commits". The Prior-art escape-hatch wording (skipped — test additions for existing capability) is correct but the substance-arm calibration window (warn-only through 2026-06-10) surfaces it as a warning. Not a blocker; worth noting for the CLAUDE.md "capability commit" definition clarity (whether ≥80 LOC test files should be excluded by the hook, not just by prose).
Hook has no @cc-only-rationale or @dual-pair marker — per dual-implementation-discipline.md §9, inject-session-bootstrap.sh is listed as one of the 4 hooks that predate the rule and lack markers. The dual-implementation-discipline.md §3 explicitly cites this hook as an example of "Internal tooling — default: CC-native only". Forward-going annotation rule applies: marker should be added at next substantive touch to the hook itself, not in this test-only PR.

Test plan

vitest green for new test file: 9 passed, 0 failed
mutation-sanity verified via /tmp copy — mutant breaks PAIRED-NEGATIVE assertion
CI green (auto)

chore: resync staging→main (promote 38 PRs + 3 merge-commits)

…p.sh (M.4.3) UserPromptSubmit hook — always-on bootstrap digest injection via plain stdout. 9 tests assert: non-empty output, opening/closing sentinel tags, content markers, bounded output (open before close), exit-0, repeat-invocation identity (no false cache), and UserPromptSubmit plain-stdout vs PostToolUse JSON channel distinction. Reference pattern: check-hook-marker.test.ts (vitest + spawnSync). Mutation-sanity verified via /tmp copy: mutant (empty-output) fails the PAIRED-NEGATIVE "output MUST NOT be empty" assertion. Production hook unchanged. Prior-art: skipped — test additions for existing capability (Track M.4 kickoff §3).

… detection) (#220) Run `bash .claude/skills/meta-orchestrator/helpers/plan-currency-check.sh` (L2 Stage 3 detection shipped in #217) → 88 UNTRACKED-N entries surfaced between the 2026-05-22 reconciliation and origin/staging tip (#217). Map each to an existing §0 / Track row by adding the PR number to its evidence cell, or to a new §0 row for two umbrellas that landed in full since the prior snapshot. Re-run helper → 0 UNTRACKED remaining. Key changes: - Snapshot date 2026-05-22 → 2026-05-25 (header + §0). - N8 row: A-phase 🔲 → 🟡 — C1 SSOT-existence (#170), C2 kickoff T-enumeration floor (#174), C3 principle 13 §1.7 substance (#178), C4 delivery-channel marker (#177), activation #180. C5 + cost-levers remain gated on §5.3 utilisation trigger. - Track M.1 / M.4 → DONE: M.1 codified T20 via #212 (with NB note — recommendation-laziness took the T20 slot, mutation-equivalence T-bump 20→21 still pending); M.4 6 paired-negative bash-hook tests shipped #195/#196/#197/#198/#199/#200. - Two new §0 rows: Meta-orchestrator skill (Track P) — BUILD #186 + audit rounds #192/#193/#194/#201/#202 + UX refactor #203/#204/#205 + planner-completeness #213/#214/#217 + §1.7 PR-body mandate #216; Recommendation-laziness discipline — R-phase #206/#207, benchmark #210, I-phase Sub-waves A/C/D #211/#212/#215. - N7 row: + dogfood research-patch #135 / §4 demotion #166 / live-trial verified #171. N4b row: + design #136 / record #118. - Infra paragraph: PR refs for I.1 follow-ups (#121/#123/#124/#125/ #128/#130/#131/#143/#145/#146/#147/#148/#149/#172/#187), I.2 (#139/#142/#154/#175), I.3 DN-4 (#126/#132/#133/#138/#140/#152/ #159/#161/#162/#167). - Track 2.3 (channel-earliness audit) → DONE 2026-05-23 (#181); removed from "What actually remains". - Footer subsection: standalone work (#191 satellite-arch / #189 guard-liveness / #173 storm-readiness / #176 §10 port / #182 cleanup), Wave 10 follow-ups (#110/#112/#113), plan-revision history (#108/ #109/#153/#155/#157/#160/#164/#165/#168/#179/#185). Verification: - `bash .claude/skills/meta-orchestrator/helpers/plan-currency-check.sh | grep -c '^UNTRACKED'` → 0 (down from 88). - `npx markdownlint-cli docs/meta-factory/wave-sequencing-plan.md` → no violations. - `npx vitest run packages/core/skills/plan-currency-check.test.ts` → 14/14 passed. Prior-art: skipped — chore, doc reconciliation only, no new capability or rule introduced.

…rix + CI firing (python-backend-v0 S1) (#960) * feat(backends): astgrep Python YAML renderer — syntax-class routing + FF7001/FF7002 refusals (S1 Task 1) python-backend-v0 umbrella, S1 Task 1. Adds packages/core/backends/astgrep/: - render-astgrep.ts: pure renderer, backend name astgrep-python-yaml, ToolchainBackend<string> conformance (astgrepBackend), mirroring backends/cargo/render-clippy.ts seams. Routing is the INVERSE of cargo: selectorClass 'syntax' renders (native per-rule severity — defaultSeverity projected directly, NO FF7003 degrade path); 'type-aware' refused FF7001 (mypy backend deferred); 'dep-graph' refused FF7001 (import-linter backend deferred); malformed/missing params refused FF7002. Params contract: {kind: 'call'|'attribute'|'import', pattern, replacement?}. Rule message is ALWAYS node.claim (render-clippy.ts:112 parity). Artifact: ast-grep YAML rule-set string (header + one doc per rule, ---separated, id-sorted). Emission shape live-verified against @ast-grep/cli@0.44.1 (scan --json: ruleId identity, exit 1 on error findings, exit 0 clean). - test-fixtures.ts: FIXTURE_NODE = ban datetime.datetime.now (the P5 live-fired case, research-patches/2026-07-02-multi-toolchain-generalization.md:409-423). - render-astgrep.test.ts: 15 unit tests (TDD red-first) — T15 grammar-gate self-application, FF7001/FF7002 paired negatives, severity projection incl. note→hint, byte-for-byte golden, YAML shape + escaping + sort. SSOT: appends rows #212 (ast-grep as Python rule-emission target, ADOPT), #213 (mypy, DEFER — trigger: first type-aware Python node), #214 (import-linter, DEFER — trigger: first dep-graph Python node) to docs/meta-factory/prior-art-evaluations.md (append-only §3 procedure). Tests: npm --prefix packages/core run test:backends (9 files / 80 tests green) + test:principles (33 files / 303 tests green). Prior-art: prior-art-evaluations.md#212 (ast-grep @ast-grep/cli@0.44.1 as the Python syntax-class rule-emission target, ADOPT — #199 render-target lineage, #185 distinct search-surface capability, P5 live-fire evidence). * feat(backends): astgrep firing harness — fixtures + shared JSON-array parser + capability matrix (S1 Task 2) Firing fixtures (invalid/valid/valid-clean) + firing-contract.json + live-fire firing.test.ts + self-application drift gate + shared JSON-array diagnostic parser + capability-matrix.json with live-fired evidence, for the astgrep-python-yaml backend. - Fixtures: canonical rule bans datetime.datetime.now (FIXTURE_NODE). invalid calls it; valid wraps it in an injected-clock accessor + the `# ast-grep-ignore: <rule-id>` suppression escape hatch; valid-clean is conformant with zero findings. Each fixture's committed rules/no-datetime-now.yml is byte-identical to render(FIXTURE_NODE) — the always-on self-application drift test gates it (runs in CI, no tool needed). - firing.test.ts: presence-check + loud-skip when the pinned ast-grep is unavailable, but NO `!isCI` guard — CI fires for real (owner decision). Live-fire RED (invalid -> ruleId no-datetime-now) / GREEN (valid + valid-clean -> zero findings). - Shared backends/shared/json-array-parse.ts: parseIdentitiesFromJsonArray for the JSON-array diagnostic shape. Justified per the kickoff §1 two-consumer condition — ast-grep (S1) and ruff (S2) both emit JSON arrays with a flat identity field. The cargo NDJSON parser is untouched. - capability-matrix.json: syntax=yes (live-fired evidence, ruleId identity matches the firing contract), type-aware=no FF7001, dep-graph=no FF7001. Toolchain-freshness gate derives the resolving ast-grep version at run time (no version literal in code) and turns RED on a pin bump without evidence regen. Live-fired against @ast-grep/cli@0.44.1 (2026-07-11): invalid -> 1 finding ruleId no-datetime-now severity error (exit 1); valid + valid-clean -> [] (exit 0). Review carry-forwards closed empirically: (m1) a 2-rule `---`-separated multi-document YAML fired BOTH rules (no-datetime-now + no-os-system) — multi-doc parses; (m2) a `severity: hint` finding DOES surface in `scan --json` default output (exit 0, not 1), so the Task-1 note->hint mapping is safe (findings visible, just non-erroring). ast-grep invocation detail encapsulated in fireContract: `scan` anchors its project root to the enclosing git repo, so both `-c <config>` and the scan target are passed as absolute paths under the fixture dir to scope the scan to that fixture's src.py. Prior-art: prior-art-evaluations.md#212 (ast-grep as Python rule-emission target, ADOPT — this commit ships the S1 firing harness + capability matrix for that emission target; SSOT row added in Task 1 commit fb07a5f). * ci(backends): astgrep live-fire via pinned PATH install — bare-binary firing model (S1 Task 3) Wire the astgrep-python-yaml backend's live-fire to fire FOR REAL in CI, and switch the firing model from an npx-resolved pin to a bare PATH binary backed by a hard, exact-pinned CI install (owner binding decision, Task 2 review follow-up). - firing-contract.json `command`: `npx -y -p @ast-grep/cli@0.44.1 ast-grep scan --json` -> `ast-grep scan --json`. The tool is NOT a package.json dependency; it comes via a workflow global install. deriveToolVersion/fireContract token handling is form-agnostic (splits on the `ast-grep` bin token), so no logic change was needed — only the recorded command and the surrounding rationale comments in firing-runner.ts / firing.test.ts / capability-matrix.test.ts. - audit-self.yml (principles-meta-tests job): add `npm install -g @ast-grep/cli@0.44.1` BEFORE the `test:backends` step. Exact pin, no float, no `latest` (ci-tool-pinning.md Rule A); no secrets, no third-party action (zizmor/no-paid-llm clean). Rationale (owner STOP-line): an npx-resolved pin converts a registry flake into a loud-skip — CI goes green without ever firing the backend. A hard install step fails loud-red instead, keeping "CI fires for real" honest. Kickoff: "Tool comes via workflow install, NOT a package.json dependency". No `!isCI` guard is re-introduced (cargo firing.test.ts:31-49 loud-skip shape, kept for the local-absent-tool path only). Evidence unchanged: capability-matrix.json still claims `ast-grep 0.44.1`; a live `ast-grep 0.44.1 scan --json` run against the invalid fixture reproduces the committed capturedDiagnostic byte-for-byte (byteOffset 38-61, ruleId no-datetime-now, severity error) modulo the already-normalized repo-relative `file` field — so no matrix regeneration was required. Verified: `npm --prefix packages/core run test:backends` green with the 3 live-fire tests FIRING (0.44.1 on PATH; 108 passed) and loud-skipping when the binary is absent (3 skipped + drift block green); `test:principles` 303 passed. Prior-art: prior-art-evaluations.md#212 (ast-grep @ast-grep/cli@0.44.1 as the Python rule-EMISSION target, verdict ADOPT — this commit wires that ADOPTed backend's live-fire into CI via a pinned PATH install; no new capability, modifies existing astgrep backend files + the CI gate). * ci(backends): suppress zizmor adhoc-packages on the pinned ast-grep global install zizmor 1.26.1 (CI-pinned) flags any npm install outside a lockfile; a global CLI tool has no lockfile surface and the exact @0.44.1 pin is the determinism mechanism (ci-tool-pinning.md Rule A). Inline-ignore precedent: demo-regen.yml:62. Verified clean against zizmor==1.26.1 locally. Prior-art: skipped — CI suppression comment only, no new capability --------- Co-authored-by: t <t@t.co>

artyhoo and others added 2 commits May 24, 2026 00:34

Merge pull request #187 from Yhooi2/staging

bb3ecec

chore: resync staging→main (promote 38 PRs + 3 merge-commits)

artyhoo enabled auto-merge (squash) May 24, 2026 11:34

artyhoo merged commit f0d457b into staging May 24, 2026
22 checks passed

artyhoo mentioned this pull request Jul 3, 2026

feat(backends): MT S2 — cargo clippy.toml renderer + firing harness (cargo-backend-v0) #886

Merged

artyhoo mentioned this pull request Jul 11, 2026

feat(backends): cargo severity projection + honest cargo clippy demo (launch-preannounce-track S4) #977

Merged

Navigation Menu

test: paired-negative tests for inject-session-bootstrap hook (M.4.3) - #199

test: paired-negative tests for inject-session-bootstrap hook (M.4.3)#199
artyhoo merged 2 commits into
stagingfrom
test/m4-inject-session-bootstrap

artyhoo commented May 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

artyhoo commented May 24, 2026

Summary

What's tested

Mutation-sanity

§1.7 Forward-check applied

§1.7 Backward-check applied

Observations (NOT fixed in this PR per T5)

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant