docs: Phase 8.8 — prior-art evaluation mechanism (planning hook) - #9
docs: Phase 8.8 — prior-art evaluation mechanism (planning hook)#9artyhoo wants to merge 5 commits into
Conversation
…tory between Phase 8 close and Phase 9 entry) Phase 8.8 = documentation/process phase mirror of Phase 7.5 pattern. Sequential, post-Phase-8-acceptance. Numbering 8.8 (not 8.5 — slot reserved per Art's explicit direction; 8.X parallel = self-diagnostics). Blocks Phase 9 entry — Phase 9 implementation risks reinventing existing solutions без structural prior-art consult mechanism. Scope brief (full design lives in Phase 8.8 session): 5-level construction. - L1 — prior-art-evaluations.md SSOT (table-driven, ≥3 initial entries Autogrep / Netlify framework-info / fitness functions vocabulary + what surfaces in Phase 8 capability decisions) - L2 — §5.5 Step 1.5 mandatory consult gate - L3 — phase entry prompt template Task 0 - L4 — CLAUDE/CONTRIBUTING build-vs-reuse invariant + Prior-art commit trailer convention + pre-push WARN hook - L5 — periodic staleness refresh routine + CI gate (Phase 8.X+ tooling) Retroactive audit clause: Phase 8.8 retro documents post-hoc consult for each Phase 8 build-vs-reuse decision; gaps where Phase 8 reinvented existing solution are explicit findings. Motivation: Phase 4-7 retros + Phase 7.5 analog research (2026-05-08) surfaced 3 closest prior-art candidates not in §3.3 — without enforced consult mechanism each future phase risks the same skip pattern.
…sult to first-class framework principle Extends Phase 8.8 scope (5 → 6 levels) with Уровень 0: promote prior-art consult from process discipline to framework-level invariant via: - New principle entry in skills/rules-as-tests/references/overview.md (canonical principles SSOT per self-application.md §2 row L2): «before building/introducing dependency, MUST run context7 query (≥3 phrasings) AND consult prior-art-evaluations.md; document via Prior-art: trailer or phase-N-research.md citation». - Matching factory/rules-manifest.json entry with check.type=manual. - Inherits SSOT pipeline benefits: catalogued, RULES.md auto-render, Phase 2 meta-test drift detection. - Equal footing с «every rule has executable check» / «AST > grep» / «no tautology». - Consumers inherit via RULES.md emission (when first consumer ships). Atomic commit count for Phase 8.8 session bumped 6-7 → 7-8.
…s-reuse passes its own framework gates Extend Phase 8.8 description with explicit recursive self-validation section. P-build-vs-reuse principle is subject to the same gates as every other rule in the framework (recursive thesis per aif-comparison.md §10 differentiator #4). Three forms of recursion documented: 1. L4 6-gate validation table — gate-by-gate status for the new principle. Gate 1 (schema) ✓ — manifest entry; Gate 2 (rule-tester) n/a — manual rule (G2 Pages-Router precedent); Gate 3 (mutation) n/a v1; Gate 4 (tautology) ✓ MUST — pre-push hook companion must fire when violated; Gate 5 (two-AI review) DEFER per §13.10 entry #4; Gate 6 (cross-rule conflict) ✓. 2. Self-application at creation — chicken-and-egg resolved: the 2026-05-08 analog research session IS the consult evidence for creating P-build-vs-reuse. Strongest form recursion: rule about prior-art consult is itself prior-art-consulted. Documented in Phase 8.8 retro as «self-application at-creation evidence». 3. 6 Уровней as defense-in-depth — each level catches a distinct class of violation failure mode (drift / staleness / phase-entry / session-start / implementation-time / time-decay). Violation must pass all 6 layers undetected to slip through. Mutation testing note: when gate 3 activates (Phase 9+ Path B), invert pre-push trailer regex → meta-test must fail. Otherwise the test of the test is itself tautological. Acceptance criteria extended: P-build-vs-reuse passes applicable L4 gates + self-application-at-creation evidence documented in retro.
…eat + Task ordering + capability commit definition + Gate 4 independent meta-test Reframing: Art = первый реальный consumer (2026-05-08), framework используется для собственных проектов. §1 «no-consumers caveat» — CLOSED. Recursive-self-validation thesis применяется в полную силу; Phase 8.X / 8.8 — больше не v2 deferred, а v1 invariants. Four substantive fixes to Phase 8.8 description: 1. Time estimate honest — 1 day → 2-3 дня; commit count 7-8 → 8-10. Документная база + Уровень 4c hook impl + Уровень 5 CI gate skeleton + recursive validation tests = day-and-half realistic, not «1 day doc phase». 2. Task ordering subsection — explicit T1-T11 sequence resolves chicken-and-egg structurally: T1 SSOT skeleton → T2 first entry (Autogrep, with Prior-art trailer ссылающийся на 2026-05-08 research) → T3 P-build-vs-reuse principle created через consult of existing T2 entry → T4-T5 more entries → T6-T11 levels + retro. Self-application at creation = T3 commit body, не post-hoc retroactivity. 3. Capability commit — formal grep-based definition. Detection logic: new dep в package.json OR new file ≥50 LOC under packages/core/<new-dir>/ OR new file ≥80 LOC anywhere в packages/. Thresholds — initial guesses per §5 numerical thresholds caveat; tune via Phase 8.8 retro on actual false-positive rate (>30% = review thresholds, не drop layer). 4. Gate 4 independent meta-test spec — tests/principles/p-build-vs-reuse-tautology.test.sh. 4 sub-tests: positive (trailer present → exit 0), negative (no trailer → warn), mutation 1 (invert regex → positive must fail), mutation 2 (drop detection → negative must fail). Tests instrument hook через subprocess в isolated tmpdir — independent от pre-push hook self- call (closes the «Gate 4 circular» concern). New CI job framework-self-build-vs-reuse depends on principles-meta-tests. Net effect: Phase 8.8 description now production-grade for first- consumer use, not minimal POC handwave.
…ifest) + process discipline (CONTRIBUTING) — resolves «выбивается from остальных» concern User concern (2026-05-08): single P-build-vs-reuse rule mixes artifact validation (research file content) с process discipline (commit metadata, developer behavior). Manifest schema fits artifact rules; check.type=manual для process part = dodge, not fit. Concern is real: existing rules check static artifacts (R-rules on consumer code, IR on manifest, meta-principles on framework files); P-build-vs-reuse checked dynamic process — categorical mismatch. Resolution — split into 2 entities by surface: (a) IR7-research-cites-prior-art (ARTIFACT rule) → promoted to manifest cleanly: - applies-to: docs/meta-factory/phase-*-research.md - check.type: command (validation script) - Negative corpus: research files без citations OR с broken refs - Passes Gates 1/2/4/6 ✓ MUST; 3/5 n/a/DEFER per §13.10 - Real artifact rule, не decoration. Subject to Phase 2 meta-tests on principle drift; rendered to RULES.md - Lives в overview.md as canonical principle (equal footing с every rule has executable check / AST > grep / no tautology) (b) Process discipline `Prior-art:` commit trailer → NOT in manifest. Lives в CLAUDE.md + CONTRIBUTING.md. Enforced via: - .husky/pre-push HARD FAIL с structured escape hatch (Prior-art: skipped — <rationale> permitted; absent = exit ≠ 0) - PR review checklist (Prior-art trailers verified) - Independent mutation-tested meta-test (anti-tautology guard for the hook itself) Different surface (process, not artifact), different enforcement model (developer-time hook + PR review, not L4 validator). Hard fail with escape hatch, not soft warn — soft warn = self-defeating если правило важно. Friction fires только на capability commits (≈1 per 5-10 commits), не каждый. Updated sections: - Уровень 0: split into (a) IR7 promoted + (b) process discipline staying в CONTRIBUTING. Rationale: framework identity preservation. - Уровень 4: HARD FAIL spec с escape hatch syntax (Prior-art: skipped — rationale). - Recursive self-validation: split into (a) IR7 L4 6-gate validation table + (b) process discipline 4-axis table (PR review + local hook + independent meta-test + periodic staleness). Different axes, intentional design boundary. - Task ordering T1-T12: T1-T2 SSOT first, T3 IR7 artifact rule, T4-T5 more entries, T6-T7 phase entry mechanics, T8-T9 process discipline (CONTRIBUTING + hook), T10-T11 meta-tests + CI, T12 retro. No chicken-and-egg для either part. - Gate 4 meta-tests: split into (a) IR7 corpus extension to existing gate-tautology.test.ts + (b) prior-art-trailer-hook.test.sh для process discipline. Two CI jobs framework-research-cites-prior-art + framework-prior-art-trailer-hook. - Acceptance: explicit split criteria (IR7 passes manifest gates; process discipline has hard-fail hook + 5 mutation-tested cases + PR template checkbox).
|
C-park per ultraview verdict (2026-05-08). 5 commits = iterative architectural pivots, не build history. Multi-agent review confirmed M1/M2/M3 (IR namespace mismatch, manifest scope violation, Phase 2 principles directory is right home) + 10 new findings (notably N1 «Mirror Phase 7.5» false claim, N3 ghost test path, N5 aif-evolve overlap missed, m4 broken bash heuristic). Threading 6+ revision commits through wrong architectural foundation > clean restart with verified findings as input. Sunk cost = 5 doc commits на ветке (preserved locally for design exploration reference, not shipped infra). Replacement: fresh Phase 8.8 session prompt (separate PR) incorporating all verified findings + new findings + reviewer recommendations:
Branch docs/phase-8.8-planning-hook stays local for reference; not deleted. |
… cleanup PR #11 review M1: globToRegex emitted `^src/app/.*/[^/]*\.tsx$` for `src/app/**/*.tsx`, requiring at least one directory between `src/app/` and the leaf file. Files at the base (src/app/page.tsx, src/app/layout.tsx) were silently excluded from glob expansion, undercounting glob overlap when comparing real divergent plans. Phase 8 acceptance trivially held at similarity=1.0 (regen vs frozen are identical inputs, so the bug was invisible to canonical-regen.test.ts), but Phase 9 entry research (Path A LLM gen ROI scoping) consumes this metric on divergent plans where the bug would understate glob overlap. Fix: special-case `**/` as `(?:.*/)?` in `globToRegex` (+4 LOC). Tightened `'glob-overlap edge'` test to assert exact 3/5 ratio (was loose `>0 <1`, would have passed with broken regex). Added regression guard `'**/ matches zero or more directory segments'`. PR #11 review M2: removed redundant globs in three new recipes: - next-r12-no-server-imports-in-client: dropped src/app/**/*.tsx and src/components/**/*.tsx (subsets of src/**/*.tsx) - next-r14-require-form-safe-parse: dropped src/app/**/actions.ts (subset of src/app/**/*.ts) - next-r20-require-use-server-directive: same cleanup as r14 Frozen canonical-v15 snapshot regenerated from updated recipes; canonical-regen test still green at similarity=1.0. Phase 8 retro updated: verification block #7 (LOC 84 -> 88), Created/modified files block (84 LOC -> 88 LOC, 9 tests -> 10 tests), Self-reflection #2 appended with post-fix bump note, new Self-reflection #9 documenting the PR #11 review fix and the meta-lesson (acceptance test invariance under self-comparison is necessary but not sufficient for metric correctness). Verified: 239/239 tests pass, make self-audit green.
…cleanup PR #12 review M1+M2+L2. M1 — broaden semver-prefix coverage in pa_is_new_dep_added: Old regex: `"\^?[0-9]` (caret or digit-prefix only). New regex: `"(\^|~|>=?|<=?|=|[0-9*])`. Now catches: `"^X"`, `"~X"`, `">=X"`, `"<=X"`, `">X"`, `"<X"`, `"=X"`, digit start, `"*"`. Still misses pure dist-tag versions ("latest" / "next" / "beta") and URL specifiers ("file:" / "git+" / "github:") — documented in the function comment as a known limitation. Rationale: scoping to broader patterns (any `[^"]+` value) would false-positive on package.json `scripts` entries which also follow the `"name": "value"` shape. Phase 9+ entry research can tighten via jq-based JSON parse if observed in practice. M2 — bump-vs-add differentiation: Pre-fix: any `+` line matching the dep-shape regex flagged the commit as capability, including pure version bumps where the same key appears in both `+` and `-` lines. Post-fix: extract dep keys from both `+` and `-` lines via grep+sed; a key is "newly added" only if present in `+` but absent in `-`. Pure bumps no longer false-positive into capability detection. Implementation discipline: standard `for key in $added_keys` loop + `printf '%s\n' "$removed_keys" | grep -qFx "$key"` membership test. No subshell-return / pipe-while patterns (PR #9 m4 finding stays enforced). L2 — remove unused `stripped` local var declaration in pa_check_trailer (leftover from earlier draft of the placeholder check; superseded by the word-tokenizer loop). T9 hook test — added 2 sub-tests to validate the M1/M2 fixes: Test 7 (M2): seed an existing dep, then commit a version bump without a trailer. Hook must NOT flag as capability (exit 0). Test 8 (M1): commit a new dep with tilde-prefix version (`~1.0.0`) + valid trailer. Hook must flag as capability AND accept the trailer (exit 0). Pre-M1-fix would have missed the tilde and silently passed the commit through. Plus 2 new helper functions (seed_existing_dep, bump_existing_dep, add_capability_commit_with_tilde) to keep sub-test bodies thin. Verified locally: bash -n on both files clean; 8/8 sub-tests pass (original 6 still green + 2 new for M1/M2); principle 08 still 7/7; make self-audit green (34/34 across all 8 principles). Prior-art: prior-art-evaluations.md#1 (PR review tightening — same Autogrep entry rationale; bump-vs-add differentiation is mechanical refinement of an existing detector, not new capability area, no separate analog research needed).
…pointer + m3 forward-finding) Closes 1 MAJOR + 2 MINOR review findings on Phase 9 prompt drafting: - M1 — A8 «calibration» framing was overclaim. PHASE-9-PROMPT.md §4 T3 said weights become «data-backed» — but 5 mutants × 3 dimensions ≈ 1 sample per dimension = regression guard for the formula, NOT statistical calibration of 0.40/0.40/0.20 values. Updated §4 T3 «Why» paragraph + sub-task #4 doc-note text + verification grep probe (regression guard, not data-backed) to distinguish «formula behaves as designed» (what corpus proves) from «weight values are right» (still requires real divergent-plan data per Phase 8 retro Self-reflection #9). - m2 — retro Versioning section did not point to next-step push + gh pr create. Added explicit «Next:» line at end of Versioning + 2026-05-09 entry recording T6 review delta-fix landing. Also corrected the branch-name reference (drafting work is on docs/phase-9-prompt-draft after Phase 8.8.1 T0 split, not on docs/phase-9-coverage-gap). - m3 — drafting session did not apply (yet-to-be-formalized) Phase 8.8.1 search-coverage methodology to its own decisions (own-stack sweep on A8 corpus shape; adversarial check on N=15; surfaced Open Q from drafting itself = 0). Circular: Phase 8.8.1 not yet merged → methodology not formally required. Documented as Open Q #4 forward-finding for post-Phase-8.8.1-merge audit. Reviewer-flagged BLOCKER B1 (drafting work «lost») was misdiagnosed — drafting commits live on this branch (origin synced at 1c422cd); T0 of Phase 8.8.1 meta-prompt split correctly. Reviewer's coverage was bounded to one branch, missed the parallel docs/phase-9-prompt-draft. PHASE-9-PROMPT.md = 307 LOC ≤500 cap unchanged; retro 119 → 124 LOC (+5 from m2 closing + m3 Open Q #4) ≤200 cap.
…ede #81), 3 branches + visible marker (#89) #81 (cfa28a3, on main) delivered the recap INSTRUCTION via systemMessage, which is user-only and never reaches the model — verified dual-channel (CC hooks docs Stop decision-control: "reason must be provided for Claude to know how to proceed" + WebSearch). So #81's recap prompt was never seen by the model. d695ac5/redesign correctly use `reason` (injected; agent gets one more turn). This merges best-of-all-three: - reason-delivery (model receives) + short systemMessage human note [supersede #81 delivery bug] - forced visible «## 🟢 Простыми словами» header [from #81 — human scannability] - session-goal anchor (aiTitle→first-msg fallback) + 3-way drift verdict + recommendation-first [from redesign] - 3 branches incl. combined long+question (keeps recommendation-first when a fork is on the table) - stateless-fire preserved (no /tmp, no aggregation/isMeta) Tested 2026-05-21: bash -n; branches A/B/C; silent on bare tool_use; AskUserQuestion→B; anchor aiTitle-primary + first-user-msg fallback + graceful default; §1.8 skip smoke-test exit 0. Prior-art: skipped — modifies existing <50-LOC-logic CC hook (delivery-channel merge), no new capability per dual-implementation-discipline §2(iv); SSOT #8/#9/#20 cover the anchoring/hook family.
Maintainer correction 2026-05-26 caught 4 invariant-level violations in first draft of B3 sub-mechanism: 1. CI cast as primary evaluator (violates README CI=last-resort invariant) 2. Literal "halt-on-first-CI-red" despite REVISE=3 in same kickoff row 3. False claim "companion-projects only in memory" (§1.13 codified 2026-05-22) 4. "Oracle via subagent = expensive/complex" while session was using it for cold-review (91k tokens / 3 min / caught 2 MAJOR) All four are nested T-traps on self (T3 + T16 + T11/T13 + recursive-self- application gap). Cold-review subagent missed all four because its prompt was structural-check, not invariant-check. Patch changes: - §0: add T-BA-E #ci-as-primary-evaluator trap (specialisation of T3+T16); enumerate it alongside existing T-BA-D #bundle-name-match - §3 B3 table: add 4 new candidates with corrected scoring — oh-my-openagent (#8, ADOPT three-attempt+Oracle protocol, ~95% match, EXACT problem class), Aider --auto-test (#9, ADOPT success-path, ~85%), OpenHands AgentController (#10, ADOPT VOCABULARY + REFERENCE, ~80%), Claude Task Master autopilot (#11, ADOPT pause-not-halt semantics, ~80%); re-score Anthropic evaluator-optimizer ~40%→~90% with reframe (Oracle subagent IS our evaluator-LLM, not CI) - §5 B3 verdict: rewrite from "ADOPT SDD + vocab from Argo/Anthropic" to ADOPT composition of 4 production patterns (oh-my-openagent + Aider + Task Master + Anthropic evaluator-optimizer + Superpowers SDD reviewer-loop); BUILD-delta ≤130 LOC (per-item-in-bundle wrapper + Oracle prompt template + bash glue), only what no upstream provides - §7 forward-check: explicit T-BA-E listing + meta-insight section acknowledging this R-phase demonstrated the Oracle layer is missing in our own workflow (not just for B3); the very pattern this patch recommends building for B3 is the pattern this patch itself needed for its own discipline check Companion memory codified: - feedback_oracle_via_subagent_reuse_existing.md (4-checker before BUILD) - feedback_cold_review_must_check_core_invariants.md (invariant-check template for future cold-review subagent prompts) - feedback_b3_revise_not_halt_on_ci_red.md (REVISE semantics correction) - feedback_ai_doc_research_priority_pool.md broadened scope (added runtime side; corrected oh-my-openagent canonical URL) Research-only: zero SSOT row mutations, no helpers shipped. Per-sub-mech verdict unchanged for B1/B2; B3 verdict substantially strengthened by prior-art composition. Stage 2 maintainer GO/DEFER/DROP decision unblocks Stage 3 I-phase. Prior-art: skipped — research-patch only, no capability commit
… consolidation + F.6 self-contained rationale Two of three MINOR findings from PR #261 Phase -1 cold-review applied (the third — SSOT #20 line-citation gap — was a false-positive; #20 already cites :L88 at patch:79 identical to peer rows). MINOR #1 (#ai-traps-section-fragmented) — added new §4.5 "AI-traps applied (canonical + domain-specific)" consolidating the T-trap enumeration that was previously fragmented inline at §4 §1.7 line 333. Each T-number now names the specific action in this R-phase that honours it (T1 sampling floor=5 → §1.4 enumerated all 84 SSOT rows; T3 → file:line + verbatim excerpts at lines 140-143/87/93-95/101-105/109-113; etc.). T-N1/T-N2/T-N3/T-N4 listed with their honouring action too. The prior inline mention at §4 §1.7 line 333 now points forward to §4.5. MINOR #3 (#f6-supplementary-rationale-relies-on-matrix-jump) — added "Supplementary, not subsumed by F.3 (self-contained statement)" paragraph at end of F.6 section. Now self-contained explanation: F.3-without-F.6 = Bug #3 fixed by side-effect but latent landmines remain; F.6-without-F.3 = Bugs #2/#4 unfixed → REJECT; combined F.3+F.6 = defence in depth on Bug #3. Reader no longer needs to jump to coverage matrix at lines 297-307 to understand why F.6 is additive not replacement. MINOR #2 dropped — reviewer misread: #20 citation has explicit ":L88" line ref identical to peer rows (#9/#43/#68/#77). Verified at patch:79. Prior-art: skipped — markdown polish on a research-patch; no new capability; per CLAUDE.md "Refactors, doc edits, test additions for existing capabilities ... NOT capability commits."
Summary
Planning hook for Phase 8.8 — documentation/process phase mirror of Phase 7.5 pattern. Inserts Phase 8.8 description into EXECUTION-PLAN.md §6 between Phase 8 (acceptance) and Phase 9+ (Path B / niches / AIF). Zero implementation; this PR is the planning anchor only — full design (~7-8 atomic commits) executes in dedicated Phase 8.8 session post-Phase-8-close.
2 atomic commits, docs-only.
What Phase 8.8 will deliver (6-level construction)
skills/rules-as-tests/references/overview.md(canonical SSOT per self-application.md §2 row L2) + matchingfactory/rules-manifest.jsonentry withcheck.type: "manual". Inherits SSOT pipeline: catalogued, RULES.md auto-render, Phase 2 meta-test drift detection. Equal footing с «every rule has executable check» / «AST > grep» / «no tautology». Consumers inherit via RULES.md emission.docs/meta-factory/prior-art-evaluations.mdSSOT (shipped reference, ≤500 lines, table-driven). Initial entries: Autogrep, Netlify framework-info, fitness functions vocabulary + what surfaces in Phase 8 capabilities.PHASE-ENTRY-PROMPT-TEMPLATE.mdTask 0 «consult prior-art-evaluations.md» as mandatory first step for Phase N entry sessions.CLAUDE.md/CONTRIBUTING.mdbuild-vs-reuse invariant +Prior-art:commit trailer convention + pre-push WARN hook (new dep / new module ≥50 LOC without trailer).framework-prior-art-staleness(Phase 8.X+ as well).Plus retroactive audit clause: Phase 8.8 retro documents post-hoc consult for each Phase 8 build-vs-reuse decision (Next 16 detection, regen diff metric, recipe expansion strategy R12/R14/R20, gate 5 invocation mode); gaps where Phase 8 reinvented existing solution are explicit findings.
Numbering rationale
Motivation
Phase 4-7 retros + Phase 7.5 analog research (2026-05-08) surfaced 3 closest prior-art candidates not in EXECUTION-PLAN.md §3.3:
framework-info— production-ready multi-framework detection.Without structural enforcement, every future phase risks the same skip pattern. Phase 8.8 closes the gap by making «consult prior-art before building» a process gate (§5.5 Step 1.5), a commit-trailer convention, AND a first-class framework principle (Уровень 0) — not just shipped-doc text references.
Acceptance
Test plan
^### Phase 8\.8returns 1 matchУровень 0returns 1 match (new in commit 2)Related
feat/phase-8-acceptance) — implementation work which Phase 8.8 will retroactively audit