docs: Phase 9 entry research — capability matrix + verdict for Phase 9 implementation - #13
Merged
Conversation
Phase 9 entry research artifact bootstrap. Header + §2 capability area list (9 candidates) covering: A1 Path A LLM gen menu pick, A2 Autogrep re-evaluation, A3 §13.10 #1 LLM-driven research extension, A4 Gate 5 build, A5 Path B AST gen, A6 recipe duplication, A7 next/any resolution, A8 glob-overlap calibration, A9 AIF schema validation. §3-§6 placeholder pointers to T2-T5 commits; principle 08 satisfied (5 SSOT citations across §1/§2 to entries #1-#3). File at 56 LOC, ≤200 invariant headroom for T2-T5 expansion. Trigger: §13.10 entry #2 fired at Phase 8 close; SSOT #1 (Autogrep, DEFER) re-evaluation explicit per its `Trigger to revisit` field.
Match-check 9 capability areas in phase-9-entry-research.md §2 against all 3 existing SSOT entries: - #1 Autogrep (DEFER, L3 LLM-driven rule generation) — matches A1/A2. Trigger condition fired: §13.10 entry #2 (Path A LLM gen) fired at Phase 8 close. Substantive consult marker appended to Rationale; fresh context7 lookup performed in T3 (next commit). - #2 Netlify framework-info (WATCHLIST, L1 detection) — no §2 match. Trigger condition NOT fired (Phase 9 scopes LLM gen + housekeeping, not detector v2). Status: still applies. No SSOT edit. - #3 Fitness functions (ADOPT VOCABULARY) — applies if Phase 9 introduces new principle-as-test framing. Status: still applies. No SSOT edit. A3/A4/A5/A8/A9 — no SSOT match; T3 runs fresh context7 resolve + ≥3 phrasings query per area per Hard Constraint #5. Last reviewed dates remain 2026-05-08 (same calendar day as Phase 8.8 SSOT bootstrap); SSOT edit on entry #1 records the Phase 9 consult event substantively per §3 of SSOT spec. Prior-art: prior-art-evaluations.md#1 (Autogrep, DEFER — re-evaluation in progress per §13.10 entry #2 fired trigger, fresh context7 in T3). Prior-art: prior-art-evaluations.md#2 (Netlify framework-info, WATCHLIST — trigger NOT fired, status still applies). Prior-art: prior-art-evaluations.md#3 (fitness functions, ADOPT VOCABULARY — vocabulary already adopted, status still applies).
…Semgrep 3-phrasing context7 lookup against /semgrep/semgrep-docs since Phase 8.8 SSOT bootstrap (2026-05-08 morning): - Semgrep Assistant (March 2024 GA, not new) — security-focused AI rule generation, no framework-upgrade-doc source signal. - Semgrep Supply Chain — per-dep upgrade-guidance PRs, different surface from rule synthesis. - No rule-synthesis-from-docs feature shipped post-2026-05-08. SSOT #1 «new Autogrep release / Semgrep rule-synthesis-from-docs feature» trigger did NOT fire. DEFER stance carries forward unchanged from Phase 8.8 T2; no SSOT verdict change needed. Prior-art: prior-art-evaluations.md#1 (Autogrep, DEFER — refresh confirmed no new rule-synthesis-from-docs feature post-2026-05-08; verdict carries forward).
3-candidate context7 lookup for «LLM picks from curated menu» analog in production: Cursor, Continue.dev, Factory ESLint Plugin. Findings: - Cursor rules (.mdc) are AI-agent prompt-rules, not ESLint plugin selection — different surface. - Continue.dev — no ESLint-rule-pick-from-menu surface. - @factory/eslint-plugin — production hand-curated plugin (21 rules, base/recommended/frontend/backend configs) explicitly for AI-coding- agent output quality, includes Next.js-aware rules and per-rule Markdown for agent adaptation. NOT LLM-driven menu pick (rules are hand-authored), but the closest production analog to the curated menu a Path A LLM gen would select from. No production tool implements LLM-driven «picks from menu» of existing ESLint rules given a codebase context. SSOT: new entry #4 (Factory ESLint Plugin, WATCHLIST) — potential reference / partial reuse target if Phase 9+ Path A LLM gen activates and recipe inventory needs scaling beyond hand-roll capacity. Prior-art: prior-art-evaluations.md#4 (Factory ESLint Plugin, WATCHLIST — new entry recording closest production analog for Path A LLM gen curated menu, hand-authored not LLM-pick).
…5 + A9 schema 3-candidate context7 lookup against /lee-to/ai-factory (×2 phrasings) and /websites/coderabbit_ai (resolve only) — alternative AI code review tools lack formalised gate-result contract surface comparable to AIF. Findings: - /aif-review is in production with review-sidecar (model: opus override per SPEC). Phase 8 §13.11 scoping (per-plan + Opus + advisory + cached) maps directly. - aif-gate-result schema unchanged since Phase 8.8 (schema_version still 1, fields unchanged). Phase 8 Task 8.4 emission still current. - No alternative warrants a new SSOT entry — CodeRabbit/Greptile are product-side without comparable contract surface. A4 verdict outcome: REUSE AIF /aif-review + review-sidecar (already in aif-comparison §9). Phase 9 question is FP-rate-data readiness. A9 verdict outcome: REUSE — Phase 11.1 closure tail = adding schema validation against fresh AIF contract (~30 LOC). Prior-art: skipped — both areas reuse already-recorded analog (aif-comparison.md §9 strongest reuse candidates table); no new SSOT entry needed since AIF reuse is documented at the cross-ref layer.
…ry forward) 3-candidate context7 lookup for «LLM writes ESLint rule TypeScript source» analog: ts-morph (query), jscodeshift (resolve), Comby (resolve). Findings: - ts-morph: TypeScript codegen toolkit (createSourceFile, addClass, setBodyText). Path B dependency if triggered. - jscodeshift: Facebook codemod toolkit, same category as ts-morph. - Comby: structural search-replace; rewrite not generate, no fit. - No LLM-driven ESLint rule synthesis tool found — matches Phase 8.8 retro finding «no analog for LLM writes ESLint rule TS source». SSOT: no new entry. Toolkits are infra-style without non-trivial trigger condition beyond §13.10 #3 already-encoded trigger. Verdict: §13.10 #3 trigger has NOT fired (Phase 8 R12/R14/R20 were mechanical lifts of existing preset rules, no «new pattern with no existing ESLint plugin»). DEFER carries forward. Prior-art: skipped — Path B trigger has not fired, all candidates are infra-toolkits without distinct prior-art rationale beyond §13.10 #3 encoding.
…h extension context7 query for §13.10 entry #1 canonical implementation: Anthropic web_search_20250305 server tool (TypeScript SDK). Findings: - web_search_20250305 is first-class with allowed_domains / blocked_domains (mutually exclusive), max_uses, cache_control, user_location, defer_loading. Stream-compatible. Same spec name in §13.10 entry #1. - Trigger «first real consumer reports gap on non-curated framework OR Phase 8 acceptance shows curated store insufficient» has NOT fired (Phase 8 closed without gap per retros/phase-8.md Open Q #2). SSOT: new entry #5 (Anthropic web_search_20250305, ADOPT WHEN TRIGGERED) — production-ready first-party tool, no third-party alternative needed. When §13.10 #1 fires, this is canonical. Also covers A6/A7/A8 (housekeeping items): no context7 needed — internal authoring conventions and calibration data, not capability areas with prior-art analogs. Verdict: A3 → DEFER (trigger ARMED, not fired). Prior-art: prior-art-evaluations.md#5 (Anthropic web_search_20250305 with allowed_domains, ADOPT WHEN TRIGGERED — new entry recording canonical L2 LLM research extension implementation per §13.10 entry #1).
§5 final matrix: 9 capability areas, verdict + Phase 9 priority per area. 5 DEFER (LLM-bearing — all triggers ARMED or fired but ROI closes negative): - A1 Path A LLM gen — no production analog for LLM-pick of ESLint rules; Factory hand-roll is industry pattern at our scale. Refined trigger condition needed. - A2 Autogrep — SSOT #1 carries forward (T3.1 confirmed). - A3 LLM-driven research ext — §13.10 #1 trigger ARMED, not fired. - A4 Gate 5 BUILD — violates §6.0 #1, no FP-rate data per §13.10 #4. - A5 Path B AST gen — §13.10 #3 trigger NOT fired. 4 BUILD (deterministic, stop-rule-compliant — Phase 9 implementation scope): - A6 recipe duplication policy (P0) - A7 next/any/ resolution tier (P1) - A8 glob-overlap calibration (P1) - A9 AIF schema validation (P0, closes Phase 11.1 partial) Phase 9 ROI re-evaluation per §13.10 #2 closes NEGATIVE: stay deterministic-v1, ship housekeeping + Phase 11.1 closure tail. Refine §13.10 #2 trigger to «recipe count exceeds N AND new framework target requires plugin-menu pattern we don't yet ship». Acceptance regex ^\| .* \| (BUILD|REUSE|DEFER|STOP) \| → 9 matches. Prior-art: prior-art-evaluations.md#1, #4, #5 (Autogrep DEFER carries forward, Factory ESLint Plugin WATCHLIST exemplar reference, web_search_20250305 ADOPT WHEN TRIGGERED) — all three drive matrix verdicts in §5.
§6 stop-rule audit per P0/P1 BUILD area; cost projection. All 4 P0/P1 BUILD areas (A6 recipe duplication, A7 next/any/ resolution, A8 glob-overlap calibration, A9 AIF schema validation) hold §6.0 stop-rules WITHOUT amendment: - §6.0 #1 (NO LLM at runtime) — all deterministic - §6.0 #2 (NO new explicit deps) — A9 hand-rolled validator, others use existing transitive deps only - §6.0 #3 (NO yargs/commander) — no CLI surface changes - §6.0 #4 (NO Path B AST gen) — none of these are AST-based Phase 9 implementation cost projection: $0 LLM spend (all P0/P1 are deterministic). §13.11 invocation shape applies to deferred A4 only, not Phase 9 scope. §6.3 documents informational conflict surface for deferred LLM areas (A1/A3/A4/A5) — if any Phase 9.X reverses a DEFER, §6.0 amendment lands as docs-only Phase 9.5 mirroring Phase 7.5 pattern. Per Hard Constraint #8 this entry-research file does NOT preempt amendment. File size: 178 LOC (≤200 invariant headroom 22 LOC for retro pointer). Prior-art: skipped — stop-rule audit + cost projection is purely internal §6.0 compliance accounting, no new analog to record.
Phase 9 entry research close. Verdict: GO with refined Phase 9 scope. Phase 9 implementation lands as deterministic housekeeping + Phase 11.1 closure tail: - A6 recipe duplication policy (P0) - A9 AIF schema validation (P0; closes Phase 11.1 partial) - A7 next/any/ resolution tier (P1) - A8 glob-overlap calibration (P1) §13.10 entry #2 ROI re-evaluation closes negative — 5 LLM-bearing areas DEFER (A1 Path A LLM gen, A2 Autogrep, A3 LLM research ext, A4 Gate 5 build, A5 Path B AST gen) with refined triggers per §5. Verification block: docs-only branch, 9 atomic + this retro = 10 total, 9/9 conventional commits, 10 Prior-art trailer lines, 9/9 verdict rows match acceptance regex, 16/16 stop-rule cells hold for P0/P1, principle 08 green throughout, 246/246 core tests pass. Self-application: first downstream consumer of Phase 8.8 mechanism exercised; observed-zero-FP across all three enforcement layers. Two new SSOT entries (#4 Factory ESLint Plugin, #5 web_search_20250305) materially affect Phase 9 verdicts. Time-vs-plan: ≈30 min wall-clock vs 1-3h target. Same compression as Phase 4-8.8 single-session burn mode. Refined Phase 9 implementation prompt drafting is next-session work. Prior-art: prior-art-evaluations.md#1, #4, #5 (all three drive verdict matrix in §5; retro records the consult outcome and SSOT growth from 3 to 5 entries during this single downstream-consumer session).
Closes 2 MAJOR + 1 MINOR review findings on PR #13: - M1 — A3 SSOT entry #5 was added with 1 phrasing; PROMPT Hard Constraint #10 strictly requires ≥3 phrasings BEFORE adding SSOT entry. Added 2 more context7 query-docs phrasings against /anthropics/anthropic-sdk-typescript covering cache_control TTL options, max_uses budget, server_tool_use.web_search_requests counter, Usage interface token-counter independence. Updated §4.A3 with 3-phrasing log + production-tracking surface details (strengthens SSOT #5 ADOPT-WHEN-TRIGGERED claim). - M2 — A1 ROI rigor: PROMPT §6 T3 explicitly named Cody and Aider as candidates; T3.2 covered only Cursor + Continue.dev + Factory. Added Cody (Sourcegraph) + Aider lookups. Both confirm: Cody's cody.contextFilters / *.rule.md and Aider's .aider.conf.yml are AGENT-side configuration (model selection, repo filters, prompt guidance), NOT ESLint rule pick-from-menu. 5-candidate coverage reinforces «no LLM-pick-of-ESLint-rules analog in production» negative-existence claim — A1 DEFER verdict strengthened, not weakened. - m1 — retro Verification block #4 said «10 trailer lines» but actual count is 11 (T5 skipped + T6 retro trailers were missing from enumeration). Corrected to 11 = 3+1×8 across 9 commits. No verdict change in §5 matrix; ROI thesis still closes negative. SSOT entries #4, #5 unchanged. Phase 9 implementation scope (A6/A7/A8/A9 BUILD; A1-A5 DEFER) unchanged. File sizes after T7: phase-9-entry-research.md 182 LOC (≤200); retro 115 LOC (≤200); SSOT 96 LOC (≤500). Principle 08 7/7, 246/246 core tests, 9/9 verdict rows match acceptance regex. Prior-art: prior-art-evaluations.md#5 (web_search_20250305 — 2 additional context7 phrasings close M1 violation of Hard Constraint #10; ≥3 phrasings discipline now satisfied for SSOT entry add). Prior-art: prior-art-evaluations.md#4 (Factory ESLint Plugin — Cody + Aider lookups close M2 coverage gap; 5-candidate base reinforces negative-existence claim driving A1 DEFER).
artyhoo
added a commit
that referenced
this pull request
May 8, 2026
Phase 9 entry research session prompt (the trigger document for PR #13 work). Untracked at session start; not allowlisted to PR #13 branch per acceptance §7. Per Phase 9 entry retro Self-reflection #6: «PHASE-9-ENTRY-PROMPT.md was not committed to this branch. Per acceptance §7, branch is allowlisted to phase-9-entry-research.md, prior-art-evaluations.md, retros/phase-9-entry.md only — committing the prompt would FAIL allowlist. Left untracked; should be committed to main separately (out of this session's scope).» This commit closes that follow-up. Companion to merged PR #13 which shipped the actual entry research deliverables.
6 tasks
6 tasks
artyhoo
added a commit
that referenced
this pull request
Jul 3, 2026
…tion-is-not-a-mechanism rule (#892) Codifies the 3a-i anchors decision + endgame design into the MT surface (doc-only, no packages/ code): - spec A1-A7 + IR-versioning policy: anchors=node.id clarification, node-id stability invariant, IR-schema versioning policy, §5.1 surface-3 composition contract, §7 plane-embryos note + ResearchGateOutcome, §4 runner-semantics note, §8 S2 live-fire supersession - new corrections file (2026-07-02 patch stays read-only per Artifact Ownership Contract) - kickoff: S3 execution split (3b npm-on-IR / 3c extraction), Stage S4, done.md-at-S4 STOP line - new Class C rule .claude/rules/attention-is-not-a-mechanism.md + AGENTS.md rules-index row §1.7: forward-check applied — the new Class C rule (.claude/rules/attention-is-not-a-mechanism.md:3 carries `> **Class:** C` + Authoritative-for) complies with no-paid-llm-in-ci (prose-only, cold-agents session-read), doc-authority-hierarchy §2-§3 (principle 09 dynamic-covered), and build-first-reuse-default (no new capability); backward-check sweep — AGENTS.md:29 rules-index row added (principle 21 rules-autoload back to PORTABLE), corrections file docs/meta-factory/research-patches/2026-07-04-mt-patch-corrections.md:1 carries scope-slug + §1.7 (principles 10/13 green), no sibling rule superseded, and MT-MASTER-PLAN Appendix items #4-#14 reconciled — #13 IR-versioning was absent from the prompt A-list and is closed in the spec §3. Prior-art: skipped — doc amendments only, no new capability (per CLAUDE.md escape hatch) Co-authored-by: t <t@t.co>
artyhoo
added a commit
that referenced
this pull request
Jul 11, 2026
…rrides where unavoidable (#980) Fixes all 16 open Dependabot alerts (triage-first, per-alert rationale below). Two atomic commits: lock/manifest bumps; drizzle fixture pins. No new dependencies — overrides entries and in-range bumps only; not a capability commit. ## Per-alert disposition | Alerts | Package | Fix | Mechanism | |---|---|---|---| | #17-#24 (7×, incl. **critical** CVE-2026-33937) | handlebars 4.7.8 → 4.7.9 | root lock | `eslint-plugin-boundaries ^5.0.0 → ^6.0.2` in `packages/preset-react-spa/package.json:26` — 6.0.2 pins `@boundaries/elements 2.0.1` → handlebars 4.7.9. Chosen over also-available ^7.0.2 as the smaller major jump with the same security outcome. The shipped template `packages/preset-react-spa/templates/eslint.config.react.mjs` is unchanged: v6 keeps the explicitly-configured legacy `element-types` rule + `boundaries/elements` settings working. | | #4, #5 (CVE-2026-8723) | qs 6.15.1 → 6.15.2 | both locks | `overrides` in root `package.json` + `packages/core/package.json` — **unavoidable**: latest typed-rest-client (3.0.0) still pins vulnerable qs 6.15.1 exactly (verified against the npm registry). | | #27 (CVE-2026-53550) | js-yaml 4.1.1 → 4.2.0 | root lock | `overrides "js-yaml": "4.2.0"` — parent markdownlint-cli2 pins exact 4.1.1; its fixed release 0.23.0 requires node>=22 while CI runs node 20, so the parent bump is blocked. Exact spec matches packages/core's direct-dep spec (npm direct-dep override rule). | | #12 (CVE-2026-48988) | markdown-it 14.1.1 → 14.3.0 | root lock | `overrides "markdown-it": "^14.2.0"` — same markdownlint-cli2 exact-pin reason. | | #10, #13 (CVE-2026-49356) | @babel/core 7.29.0 → 7.29.7 | both locks | in-range `npm update` (parents allow ^7.x / ~7.29.0). | | #7 | esbuild 0.28.0 → 0.28.1 | packages/core lock | in-range `npm update` (tsx ~0.28.0, vite ^0.28.0). Root esbuild was already 0.28.1. | | #16, #28 (CVE-2026-39356, high) | drizzle-orm ^0.40.0 → ^0.45.2 | 2 fixture manifests + stub | `packages/core/detector/fixtures/hono-drizzle-monorepo/apps/api/package.json:5`, `packages/core/research/fixtures/tier1-single-root/package.json:3`, stub `.../node_modules/drizzle-orm/package.json` bumped in lockstep. Consuming tests verified version-insensitive (name-based detection; `research-plan.json` has `version: null`) — fixed, not dismissed. | ## Notes - **Synth bundle deliberately NOT regenerated:** the bundle is built by ROOT esbuild (`scripts/build-synth-bundle.sh:17` uses `$ROOT/node_modules/.bin/esbuild`), which was already 0.28.1 and is unchanged; the packages/core esbuild bump is a non-feeding transitive. `NODE_ENV=development build-synth-bundle.sh --check` green after fresh `npm ci` of both locks. - **Consumer note (preset-react-spa):** boundaries v6 changes the `dependency-nodes` default from `["import"]` to `["import","export","require","dynamic-import"]`. Consumers may see new (legitimate) `boundaries/element-types` findings on `export … from` / `require()` / dynamic-import edges — stricter, not broken. - **Expected `npm ls` marker:** `invalid: qs@6.15.2` under typed-rest-client is the documented override-forced exact-pin violation (the point of the fix); no CI gate runs `npm ls`. ## Verification - Full local CI-equivalent gate set green: typecheck, full packages/core suite (2288 tests, 0 failed), `format:check`, `NODE_ENV=development build:synth-bundle:check`, `render-rules --check`, `render-rule-index --check`, install-sh `gh-534-arch-boundaries` + `f17-lint-rules-planted-violation` + `byte-identical` (fingerprints unchanged — no SNAPSHOT_MODE regen needed). - Lock integrity independently re-verified from clean state: `npm ci` (root) and `npm ci --prefix packages/core` both exit 0; every touched lock entry has version+resolved+integrity; zero unexpected version changes in either lock diff (only intended packages + their transitive helper chains). ## §1.7 Self-discipline check (REQUIRED if PR touches discipline-bearing files) ### §1.7 Forward-check applied This PR introduces no new rule; the forward obligation is that each fix lands at the earliest reachable channel and is executable, not prose: vulnerable versions are excluded structurally by manifest constraints — `package.json:6-10` (overrides block), `packages/core/package.json` overrides, `packages/preset-react-spa/package.json:26` (^6.0.2 floor) — so any future `npm install` cannot silently re-resolve a vulnerable version; regressions fail at install/lock level, before CI. ### §1.7 Backward-check applied Class of this change = dependency-version-constraint updates. Surfaces where the class occurs, each swept: root `package-lock.json` (all 6 target packages patched, no residual vulnerable entries — verified over the full lock, not the diff); `packages/core/package-lock.json` (qs/@babel/core/esbuild patched; contains NO handlebars/js-yaml/markdown-it entries → no overrides needed there); fixture manifests `packages/core/detector/fixtures/hono-drizzle-monorepo/apps/api/package.json:5` + `packages/core/research/fixtures/tier1-single-root/package.json:3` + its stub (bumped in lockstep, internally coherent); other fixture manifests under `packages/core/**/fixtures/` (swept — no other pinned occurrences of affected packages); shipped templates + `install.sh` (grep: zero version pins of affected packages); synth bundle `packages/core/synthesizer/synth-and-wire.bundle.mjs` (not fed by the core esbuild entry — `scripts/build-synth-bundle.sh:17`; `--check` green); install fingerprints/baselines (`tests/install-sh/byte-identical.test.sh` green — unshifted).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 9 Entry Retrospective — Step 0 research close
Scope
Phase 9 entry was a research-only phase — zero code edits. One transient artifact (phase-9-entry-research.md, 178 lines, ≤200 invariant) plus two SSOT entries (#4 Factory ESLint Plugin, #5 Anthropic web_search_20250305) plus this retro. 9 capability areas examined; per-area go/no-go decisions recorded; Phase 9 implementation task list (4 P0/P1 BUILD areas) emerges from the matrix.
The motivation: open-questions.md §13.10 entry #2 trigger fired at Phase 8 close (Path A LLM gen ROI scoping). Phase 8.8 mechanism (T2-T11) ships as a forward gate; this session is its first downstream test — every claim cites SSOT, every capability commit carries
Prior-art:trailer, principle 08 enforces citations on the new research file.Verification block
git diff main --name-onlynon-allowlistedphase-9-entry-research.md,prior-art-evaluations.md,retros/phase-9-entry.md)Prior-art:trailers on capability commits^| .* | (BUILD|REUSE|DEFER|STOP) |≥9Capability matrix summary (compact)
next/any/resolutionPhase 9 implementation scope: 4 BUILD areas (A6/A7/A8/A9), all deterministic, all stop-rule-compliant; 5 DEFER (all LLM-bearing).
Self-application — Phase 8.8 mechanism dogfooded
Three enforcement layers active and exercised: