docs(rules): add §1.10 type-system > prose to phase-research-coverage - #63
Merged
Merged
Conversation
Adds a 10th methodology checklist item for SDK-shaped claims: when type-system evidence (e.g. agent-sdk/typescript.md interfaces, .d.ts files) diverges from prose documentation, type-system wins. Also updates §1 heading from "(9 items)" to "(10 items)" to reflect the addition. Distilled from 2026-05-16 three-channel verification incident on research-patch 2026-05-16-§17-think-time-gate.md, where Worker + Reviewer WebFetches converged on the same prose misreading of the Stop hook lifecycle. The third channel via claude-code-guide with TypeScript SDK access resolved unambiguously (StopHookInput vs SessionEndHookInput). Single-incident promotion accepted per §1 closing paragraph because the mechanism (types compile; prose doesn't) is structural, not heuristic. Source: D7 verdict A in .claude/orchestrator-prompts/d-items-strategic-dialogue/decisions.md (gitignored). See referenced research-patches in §1.10 body.
7 tasks
artyhoo
added a commit
that referenced
this pull request
May 22, 2026
…le (#139) * feat(principles): principle 17 — no paid LLM in CI (DN-6) [→staging live-test] (#132) (#133) * docs(research): memory coverage audit (memory → docs → tests) 2026-05-22 (#126) R-phase report: triages all 51 project-memory files against the project goal (rule = executable test), builds a 30-row coverage matrix by pipeline stage (0 memory-only / 1 prose / 2 executable), and proposes a forward-going memory-codification discipline (write-time + local-audit + periodic re-audit). No implementation — gap-closure + new rule are a separate PR after maintainer GO. Successor to 2026-05-13 memory-to-docs codification audit (extends memory→docs for 6 entries to memory→docs→tests over all 51 files). Surfaces T16 stale-header finding (principle tests 11/12/13 shipped but rule headers say "pending"). Prior-art: skipped — research-patch doc only, no new capability/dependency; reuses 2026-05-13 §7 REUSE verdict (Cline Memory Bank + CC scope hierarchy). * feat(principles): Wave 10.6 — port hook-stub-completeness audit to principle 16 (#127) - Add packages/core/principles/16-hook-stub-completeness.test.ts (7 tests): (a) real-tree vacuous pass: empty hard-fail set post-migration → passes, not dies (b) paired-negative: make_test_repo() test file missing stub → ❌ violation detected + message text asserted (c) non-empty happy path: stub present → no violation + scope gate, multi-script, dedup, T15 self-application arms - Delete packages/core/audit-self/hook-stub-completeness.test.sh (bash predecessor) - Remove requireSelfTest('…hook-stub-completeness.test.sh') invocation from pre-push.ts §3a - Remove requireSelfTest() helper (zero call sites remaining → genuinely dead) - Leave phantom stub in tests/hooks/prior-art-trailer-hook.test.sh (harmless; out of scope) Prior-art: skipped — bash→TS port of existing hook-stub-completeness audit (Wave 10.6), no new capability * docs(automerge): codify branch-from-main staging flow + resync discipline (#128) * docs(automerge): codify branch-from-main→staging flow + resync discipline; sync doc to LIVE state main's copy was stale (still called ci-success a placeholder). Updates: status LIVE (settings applied 2026-05-22); new §2.1 branching flow — always branch FROM main, auto-merge INTO staging, with the load-bearing RESYNC discipline (ff staging→main after each promotion) that keeps staging a disposable buffer not a divergent develop; §5 recipe marked APPLIED with the real ci-success+actionlint+zizmor contexts + main owner-only protection; §6 #2 RESOLVED (#125). Prior-art: skipped — doc codification of an already-decided flow, no new capability/dependency/subdir. * docs(§2.1): drop trunk-based exception — everything routine → staging (0 clicks) Maintainer's point: direct-to-main is owner-only → forces a manual merge click per PR, the exact toil being removed. staging auto-merges (0 clicks). So all routine work → staging; direct-to-main only for owner hotfixes. Added dependency note: for true zero-click walk-away the agent must set auto-merge on staging-targeted PRs, currently blocked by the git-safety hook (allows only feat→epic). Relaxing it (permit auto-merge --base staging) is the actual zero-click lever, and is a maintainer-side hook edit. Prior-art: skipped — doc refinement of the codified flow, no new capability. * feat(hooks): Wave 10.5 — bash-fallback + install.sh feature detection (#129) Ships the critical-only bash fallback for the pre-push hook and updates the consumer-facing dispatcher template to runtime feature detection. Artifacts: - packages/core/hooks/checks/registry.ts (~114 LOC): declarative check-registry ({ id, criticalForFallback, runner }[]) decoupling check-set selection from execution (ADAPT from Aider §4.8.X.2). Critical entries: prior-art-presence + s17-presence (both runner: 'bash'), per D2 + research patch §7.2. - packages/core/hooks/checks/registry.test.ts: unit tests asserting bash-expressible invariant (every criticalForFallback entry → runner: 'bash') AND presence of both required critical entries. - packages/core/hooks/pre-push.fallback.sh (~63 LOC): critical-only bash gate. Runs §7 Prior-art presence + §1.7 presence checks on origin/main..HEAD commits. Historical cutoff (2026-05-12) respected. bash 3.2-compatible. exit 1 if either required trailer is absent; exit 0 when both present or no discipline files touched. - packages/core/templates/shared/husky-pre-push.sh: updated from OLD consumer pre-push to runtime dispatcher (per research patch §7.4). Node ≥20 + pre-push.ts present → TS-core; otherwise → bash fallback. Capability-check, NOT brand-name. - install.sh: also copies pre-push.fallback.sh to consumer project so the runtime dispatcher can find it at $REPO_ROOT/packages/core/hooks/pre-push.fallback.sh. - docs/meta-factory/prior-art-evaluations.md: SSOT entry #59 added in-commit (registry.ts ≥80 LOC under packages/ → capability commit gate fires). T15 self-application: the registry invariant (every criticalForFallback check is bash-expressible) is itself unit-tested in registry.test.ts — the rule applies to itself. Zero edits to pre-push.ts (parallel-overlap avoidance; 10.6 owns that file). Prior-art: prior-art-evaluations.md#59 (Aider parse_lint_cmds / self.languages, verdict ADAPT — structural pattern of decoupling selection-table from execution-runner transfers; semantic axis differs: file-language vs project-stack. Wave 10.5 registry.ts is 114 LOC ≥80 threshold, capability commit gate fires, SSOT entry added in same commit per CLAUDE.md discipline). * fix(ci): make ci-success the sole required gate; fold actionlint+zizmor under it (#130) Branch protection on main/staging required ci-success + actionlint + zizmor, but actionlint/zizmor lived in workflow-integrity.yml path-filtered to .github/workflows/** — so PRs not touching workflows (docs, packages) never triggered them, their required contexts never reported, and the PR deadlocked (auto-merge could never fire). PR #126 hit this exact wall. Fix: cross-workflow `needs:` is impossible, so move actionlint + zizmor into audit-self.yml (no path filter → runs every PR) and add them to ci-success's `needs:`. ci-success now transitively gates the linters and always reports, so branch protection can require ONLY ci-success. workflow-integrity.yml keeps just the R11 branch-protection-assertion. Checks are stricter, not weaker: a broken workflow YAML on a docs PR is now caught (previously it wasn't). Synced: rules-manifest R11 check.command + re-rendered RULES.md table + snapshot; RULES.md R11 prose; ci-success-gate.sh comment; automerge-staging-plan.md §5 recipe (was telling the maintainer to re-add the deadlocking contexts). Maintainer-side follow-up: re-run the branch-protection PUT with single-context payload {"contexts":["ci-success"]} per automerge-staging-plan.md §5. Prior-art: skipped — moves existing CI jobs between workflow files + doc sync; no new capability, dependency, or ≥80-LOC file. * feat(principles): principle 17 — no paid LLM in CI (DN-6 stage-1→2 promotion) Promotes .claude/rules/no-paid-llm-in-ci.md from "Class A, grep mechanism ready, test pending" to a real executable gate (memory-coverage-audit §10 DN-6 — the one clean stage-1→2 win in-repo). Scans .github/workflows/*.yml for paid-LLM *usage* (ANTHROPIC/OPENAI api-key assignment or secret ref, api.{anthropic,openai}.com hostnames, paid SDK imports) and fails CI if any is present. Usage-precise, not mention-counting: full-line comments are stripped first, so the project's own negation mention (framework-self-template-render.yml: "NO ANTHROPIC_API_KEY reference") does NOT false-positive. Negative-controls in the test prove that, plus session/MCP tooling (claude CLI, context7) is out of scope per rule §2. Slot 17 (01-16 occupied). Runs in CI via principles-meta-tests → gated by ci-success. 7/7 local; full suite 17 files / 106 tests green. §1.7: forward-check applied — principle 17 IS the executable artifact for an existing prose rule (no-paid-llm-in-ci.md §1/§6 "pre-merge grep" counter); paired-negatives at packages/core/principles/17-no-paid-llm-in-ci.test.ts:91 (api-key assignment), :96 (hostname), :101 (SDK) prove detection, negative-controls at :107/:112 prevent false-positives. Backward-check sweep — reviewed principle slots 01-16 (17 next free) + prior-art-evaluations.md; no overlap (secret-scanners detect leaked creds, this bans a paid-LLM dependency surface — inverse problem class). Prior-art: skipped — principle test for the existing no-paid-llm-in-ci.md rule (DN-6 promotion); no new capability/dependency/subdir, mirrors principle 15/16 precedent. * docs(research): rule-enforcement channel-selection prior-art survey Survey of just-in-time rule delivery to AI agents (companion-first: Superpowers/aif-handoff/AIF/OhMyOpencode -> CC-native -> ecosystem). Validates+refines the narrowest-reachable-channel principle into a two-axis model (detectability->gate/inject; relevance->breadth). Proposes SSOT rows #60-#63; codification home deferred to maintainer (Option A). * feat(rules): rule-enforcement channel-selection (Class C) Codify the narrowest-reachable-channel principle from the 2026-05-22 prior-art survey: deliver each rule by two axes (detectability->gate/inject; relevance->breadth), reliability-ordered (deterministic matcher >= always-on > semantic > memory). Reserve always-on for 3-4 invariants; never memory for load-bearing rules. Register the rule in principle 09 REQUIRED_HEADER_DOCS so its authority header is enforced. Class C (mechanism deferred: ADAPT rule-injector hook per patch section 4). Prior-art: research-patches/2026-05-22-rule-enforcement-channel-selection.md — prior-art survey (Superpowers / aif-handoff / AIF / OhMyOpencode / Cursor / Agent RuleZ). Verdict: the meta-discipline (which-channel selection) is BUILD — no upstream rule-selection discipline to adopt verbatim; delivery mechanisms are REFERENCE (CC hooks native, SSOT #20). Proposed SSOT rows #60-#63 surfaced for maintainer, not written. §1.7: forward-check applied — rule complies with no-paid-llm-in-ci (deferred ADAPT hook is deterministic bash, not an LLM call), doc-authority-hierarchy (Class + Authoritative-for header present; registered at packages/core/principles/09-doc-authority-hierarchy.ts:43 so principle 09 enforces its header — verified test 17/17), and README earliest-reachable-channel (delivery-scope companion on a separate axis, not a conflicting goal claim); backward-check sweep — the channel-declaration obligation (rule §3 step 5) is forward-going per the §6 'Existing rules' note, parallel to dual-implementation-discipline §9; no retroactive sweep of the existing .claude/rules/*.md required, no CI gate checks channel declaration (Class C).
6 tasks
artyhoo
added a commit
that referenced
this pull request
May 22, 2026
…#154) Backfills the prior-art register for the channel-selection wave (#139): the rule + inject-matching-rule.sh hook shipped without their SSOT rows. Adds the four candidates from research-patch §5 (2026-05-22-rule-enforcement-channel-selection): - #60 Agent RuleZ — REFERENCE (Rust dep for native CC-hook capability) - #61 OhMyOpencode rulesInjector — REFERENCE + ADAPT (our inject-matching-rule.sh) - #62 Cursor rule types — ADOPT VOCABULARY (breadth ladder 1:1) - #63 agent-situations — REFERENCE (check-gated injection) Prior-art: skipped — SSOT register append documenting channel-selection prior-art, no new capability in this commit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.claude/rules/phase-research-coverage.md:57.claude/orchestrator-prompts/d-items-strategic-dialogue/— gitignored local artefact)Why now
Three-channel verification of
docs/meta-factory/research-patches/2026-05-16-§17-think-time-gate.mdrevealed Worker + Reviewer WebFetches both mis-read the same prose lifecycle table («Stop fires only at session end»). Type-system evidence (StopHookInputvsSessionEndHookInputinagent-sdk/typescript.md) was the discriminating channel — types unambiguously distinguish per-turn Stop from session-end SessionEnd.Lesson: when SDK-shaped claim's prose docs and types diverge, types win. Single-incident promotion accepted per §1 closing paragraph because the mechanism («types compile; prose doesn't») is structural, not heuristic — same precedent as §1.8 (Wave 7 M1) and §1.9 (Wave 7 M2).
Test plan
npm run -w @rules-as-tests/core test:principles— 56 tests passed (10 test files), 0 failures.claude/rules/phase-research-coverage.md:57, 16-line block including separator§1.7 Forward-check applied
Compliance with currently-active enforcement layers:
.claude/rules/phase-research-coverage.md:57(no TypeScript / JSX touched).packages/core/principles/*.test.ts): ran locally on the worktree, 56/56 pass across 10 test files. Specificallypackages/core/principles/09-doc-authority-hierarchy.test.ts:1passes —.claude/rules/phase-research-coverage.md:3retains itsdescription:frontmatter establishing authoritative-for scope (lines 3-5, unchanged).Prior-art:trailer per CLAUDE.md:14): N/A — this is a rule extension (doc edit), not a capability commit. Hook detection criteria: new package.json dep — none; new file ≥50 LOC underpackages/core/<new-dir>/— none; new file ≥80 LOC underpackages/— none. Pre-push capability detection won't fire.docs/meta-factory/research-patches/2026-05-16-claude-code-guide-cross-verification.md:402§12.6,docs/meta-factory/research-patches/2026-05-16-think-time-s17-gate-correction.md:60§4). No new external prior-art consult needed — this is methodology extension from in-project incidents, not a new capability with external precedent.docs/meta-factory/open-questions.md§13.x items in scope of this rule extension..claude/rules/doc-authority-hierarchy.md:1):phase-research-coverage.mdretains its authoritative-for declaration at lines 3-5 (unchanged); new §1.10 inherits authority from the containing file's scope statement (the «searching-layer discipline rule»).§1.7 Backward-check applied
Scope of §1.10: future R-phases that make SDK-shaped claims — hook payload fields, MCP tool contracts, settings.json schema field types, harness event interfaces, language-server APIs.
Existing artefacts in scope (complete sweep):
docs/meta-factory/research-patches/2026-05-16-claude-code-guide-cross-verification.md:402— §12.6 «Verification chain — three independent passes catch what two miss». This patch already applied the principle (§12.2 used TypeScript SDK as third channel). It is the «patch that motivated the rule» — rule would have prevented the 2-channel-convergence failure mode it documents.docs/meta-factory/research-patches/2026-05-16-think-time-s17-gate-correction.md:60— explicitly articulated «Type-system evidence (interface fields, required vs optional) is more reliable than prose evidence (lifecycle tables, narrative descriptions) when the two diverge». This is the «patch that proposed the rule».docs/meta-factory/research-patches/2026-05-16-§17-think-time-gate.md:235— original SDK-shaped patch that MISSED the principle (Stop hook claims were prose-only, no type-system check). The errata + §1.10 promotion together address this gap retrospectively per D6 verdict A.Exemption mechanism: §1.10 applies to FUTURE R-phases (those started after this PR merges). Patches pre-dating 2026-05-16 are out of scope unless explicitly re-verified; the errata pattern (D6 verdict A, recorded in dialogue decisions) handles cases where prior patches need correction.
Exemption meta-test: N/A by design — §1.10 is a methodology rule consumed at R-phase decision time, not a CI gate. Mechanical enforcement is out of scope for this rule (per
.claude/rules/no-paid-llm-in-ci.md:1— semantic «did you apply type-system check?» requires LLM); substantive compliance is reviewer-mode work viaagents/compliance-verifier.md:1extended to SDK-shape claims in future R-phases.Self-reflexive trigger: §1.10 mechanism («types compile; prose doesn't») is structural — applying §1.10 to its own justification yields the same conclusion (TypeScript SDK types for hook payloads at
docs/meta-factory/research-patches/2026-05-16-think-time-s17-gate-correction.md:37were the disambiguating evidence in the originating incident). Self-reflexive consistency holds.Pre-push hook note
Commit
0fd231bdoes not carry the§1.7:commit trailer (pre-push hook warning observed during push; calibration window currently warn-only through 2026-06-10). PR body §1.7 sections above are the binding substantive compliance evidence peragents/compliance-verifier.md:1. If the commit-trailer check graduates to binding before merge, maintainer can amend + force-push to this feature branch (no shared history yet — safe to rewrite).Source dialogue
Full decision context lives in
.claude/orchestrator-prompts/d-items-strategic-dialogue/(gitignored). Dialogue closed 13 D-items on 2026-05-16; D7 = this PR. Other 12 verdicts queued for Wave 2 (cleanup batch) and Wave 3 (skill drift detection).