Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/meta-factory/prior-art-evaluations.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ Each entry is a row in the table at §4 below. The row schema:
| 121 | Shipped-agent/skill tool-name validity — the **absence** of any tool that fails CI when a shipped CC sub-agent declares a non-canonical tool name. Surveys: own-stack liveness lineage #114 (ESLint guard-liveness gate) + #115 (manual-rule-liveness-prober) + #53 (promptfoo/Inspect/METR behavioural-eval prior art); CC upstream issues #52055 (plugin subagents never receive Grep/Glob in tool schema), #18837 (`allowed-tools` in skill frontmatter not enforced), #8697 — invalid/missing subagent tool-schema is a known real failure class. WebFetch `code.claude.com/docs/en/tools-reference` 2026-06-16 (DeepWiki unavailable in env) established the canonical allow-list. No production tool validates a shipped agent's `tools:` frontmatter against the CC-canonical name set. | M1 gate: deterministic principle test `packages/core/principles/21-shipped-agent-tools-valid.test.ts` parsing every shipped `agents/*.md` `tools:` + shipped-skill `allowed-tools` (install.sh-copied surface only); asserts each entry's base name (scope-stripped, e.g. `Bash(git *)`→`Bash`) ∈ CC-canonical allow-list ∪ `^mcp__<server>__`. Closes #551 (6 shipped agents declared `read_file`/`list_files`/`run_command` → zero-tools CC dispatch → fabrication, while `npm run validate` was green). | 2026-06-16 | 2026-06-16 | BUILD | BUILD (no upstream analog): the survey found no production tool that validates a CC sub-agent's `tools:` frontmatter names and fails CI on a non-canonical entry; CC upstream issues #52055/#18837/#8697 confirm invalid/missing subagent tool-schemas are a real, untooled failure class. Same own-built liveness family as #114 (ESLint guard-liveness) and #115 (manual-rule-liveness-prober) — the framework builds the liveness gate the upstream harness does not provide. **T16 problem-class:** distinct from #53 (behavioural eval — *did an intervention change agent behaviour*) and #115 (manual-rule RED→GREEN behavioural probe) — this M1 gate is a static **FORM-check** (do the declared names *resolve*), explicitly **NOT** a behaviour-check (a valid name ≠ the agent uses the tool; the M2 dispatch-probe is Phase 2, deferred behind a trigger). Problem class = the framework's OWN shipped delivery artefacts (agents/skills) — the `#recursive-self-application-gap` #551 exposed at the delivery layer (thesis applied bottom-up to consumer code, never top-down to shipped sub-agents). Dependency-free (vitest + `node:fs` already installed); zero new deps. CC-scoped by design (only CC's Agent tool reads `tools:`); the agnostic prose body stays harness-neutral (scope decision (i)). | CC ships a built-in validator that fails on non-canonical subagent tool names → flip BUILD→ADOPT; OR the M2 behavioural dispatch-probe (Phase 2) lands, superseding the form-check with a behaviour-check; OR the CC-canonical tool list at `code.claude.com/docs/en/tools-reference` changes → re-verify the allow-list constant. |
| 122 | `session-report` CC plugin = usage analytics (tokens/cache HTML dashboard); ecosystem session-recap skills (`annikalewis/claude-recap`, Ben Poole) = cross-session memory ("where did we leave off") | Engaging plain-language story-in-acts completion recap for the human (`/story` skill + Stop-hook branch) | 2026-06-16 | 2026-06-16 | BUILD | Composite survey verdict (operative decision = BUILD; reuse nothing). T16 problem-class check: session-report = usage/cost analytics → REJECT (different problem). claude-recap et al. = memory/continuity archaeology for the NEXT session → REFERENCE only: reusable bit is "read ~/.claude/projects/*.jsonl transcript directly", already present (end-of-turn-reminder.sh reads transcript_path). Story-in-acts style is maintainer taste → BUILD (cheap: lang-function + skill + helper + hook branch, no new dep). Survey 2026-06-16: installed companions + SSOT + WebSearch x2. | If a companion ships a human-facing engaging-completion-recap (not analytics, not memory) — re-evaluate ADOPT. |
| 123 | Coupled internal-English-machinery gate + `AIF_HOOK_LANG`-gated human-facing output localization for an AI-agent harness — surveyed for an upstream that does the *coupled* discipline (not just one half). Search 2026-06-17: DeepWiki MCP unavailable in env (per #121 precedent) → substituted WebFetch (deepwiki.com timed out) + WebSearch ×3 («lint rule enforce English-only code comments», «i18n gate AI agent output language env var locale switch», «CI fail non-English/Cyrillic source comments») and repo-level ×3 («framework gate internal tooling one language while user-facing output locale-switched», «tool fails CI on non-English comments in scripts/hooks», «env-var-gated output-language directive for an AI agent harness»). Adjacent single-purpose tools surfaced: comment-lang-detector (GitHub Action, fails CI on configured-language comments), Cyrillic Check (cyrchk, SVN pre-commit Cyrillic scan), Pulp Fiction (PyPI non-English-comment detector), blocklint (non-inclusive wording). None couples an internal-English gate that *preserves* curated bilingual match-data with an env-gated output-language directive. | The language-discipline guard: `.claude/rules/language-discipline.md` (3-category model — machinery=English / human-facing=`AIF_HOOK_LANG`-gated / category-3 match-data=kept) + `packages/core/principles/22-internal-english.test.ts` (Surface 1 machinery-shell zero-tolerance Cyrillic scan + Surface 2 SKILL.md bodies after frontmatter, allowlisted), reusing the existing `AIF_HOOK_LANG` + `lang/*.sh` pack mechanism. | 2026-06-17 | 2026-06-17 | BUILD | BUILD for the coupled guard; the surfaced detection tools are REFERENCE-class precedent for the detection *sub-pattern* only — not adopted. **T16 problem-class:** Upstream class (comment-lang-detector / Cyrillic Check / Pulp Fiction) = «flag any non-English / configured-language characters in code comments, optionally fail CI» — a blanket language-of-comments check. Our class = «zero-tolerance internal-English over AI-agent *machinery* (hook/script logic, SKILL.md bodies) that *deliberately preserves* category-3 Russian match-data (skill `description` triggers, `AIF_EOT_*` patterns, `lang/ru.sh`) AND drives human-facing prose via `AIF_HOOK_LANG`». Match? No — upstream tools have no allowlist for legitimate-bilingual match-data (would FP-storm on category 3) and no coupling to an env-gated output directive; adopting one adds a CI dependency for what is a few-line dependency-free vitest grep (cost-gate: BUILD is «cheap» per [build-first-reuse-default.md §1.1](../../.claude/rules/build-first-reuse-default.md)). REUSE half already taken: the gate reuses `AIF_HOOK_LANG` + `lang/*.sh` rather than building i18n infra ([language-discipline.md §7](../../.claude/rules/language-discipline.md)). Same own-built liveness/discipline family as #120/#121. Refs: github.com/kubecub/comment-lang-detector; cyrchk.sourceforge.net; pypi.org/project/pulpfiction. | A production tool ships the *coupled* internal-English-gate + env-gated-output-localization discipline → flip BUILD→ADOPT/ADAPT; OR Surface 1's machinery Cyrillic scan is re-evaluated standalone — then comment-lang-detector is the ADAPT candidate to weigh against the dependency-free vitest grep; OR the category-3 keep-list / `lang/*.sh` mechanism changes shape. |
| 124 | cruft (cruft/cruft, Python, built on Cookiecutter, stable 2026) | Consumer-side **re-sync of templated/vendored files** preserving local edits: `cruft update` diffs from the commit hash in `.cruft.json`; a **`skip` glob list** (`.cruft.json` / `pyproject.toml [tool.cruft]`) preserves consumer-owned files. | 2026-06-17 | 2026-06-17 | ADAPT | Surfaced by the consumer-upgrade-path R-phase sweep (WebSearch ×3, 2026-06-17; DeepWiki MCP unavailable in env per #121/#123 precedent — substituted WebSearch + SSOT consult). **T16 problem-class:** Upstream class = re-sync a Cookiecutter *template* into a project whose files the user is *expected to edit*. Our class = re-sync *framework-owned* artefacts the consumer is *not* expected to edit in place (divergence belongs in `<file>.override.md`). Match? **Partial** — the `skip`-glob (= our consumer-owned boundary) and the `.cruft.json` version stamp (= our optional version-awareness) are the transferable **concepts**; the Python runtime is **not** adopted (same vendoring blocker as #22). ADAPT the concepts into the existing `copy_safe` + `merge_prettierignore` (`install.sh:207-249`) machinery, dependency-free. Recommended path in [research-patches/2026-06-17-consumer-upgrade-path.md §3](research-patches/2026-06-17-consumer-upgrade-path.md). **The final ADOPT-vs-ADAPT dependency call is PARKED for the maintainer** (kickoff §9) — this row records the concept-level evaluation, not a finalised dependency decision. | A Node-consumable re-sync primitive emerges (then re-weigh ADOPT vs the dependency-free ADAPT); OR a consumer needs true 3-way merge of files that are *both* framework-owned and locally-edited (then #22's Copier `update` earns the Python cost); OR the maintainer resolves the §3 dependency fork. |
| 125 | Copier `update` 3-way merge (copier-org/copier, Python, v9.x) — applied to the **consumer-refresh** capability (distinct from #22's template-render-replay vocabulary) | Idempotent in-place refresh of shipped framework artefacts preserving consumer customisations: 3-way merge (old-render ⊕ user-diff ⊕ new-render), state in `.copier-answers.yml`, conflict markers/`.rej`. | 2026-05-10 (as #22) | 2026-06-17 | ADAPT | Re-evaluation of #22 against the **refresh** capability (not the test-pattern vocabulary #22 adopted). #22's `Trigger to revisit` («Copier ships answer-file-replay primitive consumable from Node without Python interpreter») was checked 2026-06-17 and has **NOT fired** — Copier remains Python-only. The 3-way-merge *core* is largely unneeded for our boundary because framework-owned files are not consumer-edit targets (the `.override.md` escape hatch carries divergence) — so ADAPT the version-stamp concept at most, do not adopt the Python dep. #22 is **not** rewritten (append-only); this row is the refresh-capability-specific evaluation. Recommended ADAPT; dependency adoption (Option B) is the **PARKED** maintainer fork per [research-patches/2026-06-17-consumer-upgrade-path.md §3](research-patches/2026-06-17-consumer-upgrade-path.md). | #22's Node-consumable-primitive trigger fires; OR the recommended dependency-free ADAPT path (research-patch §3 Option A) proves insufficient for a consumer needing partial/3-way merges → ADOPT Copier and re-open the Python-dependency cost. |

---

Expand Down
Loading
Loading