Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
cdca1d0
docs(meta-factory): Phase 4 Step 0 entry artifacts — research + prompt
artyhoo May 8, 2026
0019849
feat(detector): Phase 4 Task 1 — v1 core (manifest + config priority …
artyhoo May 8, 2026
a9eb2cc
feat(setup): Phase 4 Task 1.5 — add --dry-run flag (prereq for L1 ide…
artyhoo May 8, 2026
b6476a4
feat(detector): Phase 4 Task 2 — AIF read-side (priority 1-3) + fixtures
artyhoo May 8, 2026
be5d742
feat(detector): Phase 4 Task 3 — CLI bin + meta-factory wiring
artyhoo May 8, 2026
e1b99f1
test(detector): Phase 4 Task 4 — snapshot tests + 7 fixture trees
artyhoo May 8, 2026
eaaf6e1
ci(audit-self): Phase 4 Task 5 — framework-self-detect job (closes L1…
artyhoo May 8, 2026
b5e16b7
feat(detector): Phase 4 Task 6 — AIF skill-context write-side (touchp…
artyhoo May 8, 2026
6a2b5ff
docs(phase-4): retro + GO verdict
artyhoo May 8, 2026
f0c4e8c
docs(phase-4): post-review calibration — self-app 9→7 (M1 vacuous L1 …
artyhoo May 8, 2026
1555fb1
fix(setup): templates path for husky hooks (Phase 3.1 fallout)
artyhoo May 8, 2026
a0af459
docs(phase-4): post-fix re-calibration — L1 (a) non-vacuous, self-app…
artyhoo May 8, 2026
9f61a02
docs(aif-comparison): sync §5 touchpoint 4 status — closed Phase 4 (m1)
artyhoo May 8, 2026
4f0f226
fix(core): typecheck — exclude probes/ consumer template + ajv named …
artyhoo May 8, 2026
ef8262a
fix(preset-next-15): typecheck — exclude templates/ from local check
artyhoo May 8, 2026
cafd4f3
docs(phase-4): close retro Open #5 + log Phase 5 entry hygiene batch
artyhoo May 8, 2026
8cc6cef
feat(templates): packages/core/templates/react-next/ scaffold + git m…
artyhoo May 8, 2026
e143a7d
docs(phase-4): close retro Open #6 — react-next templates path rot (o…
artyhoo May 8, 2026
ea3e006
feat(detector): v1.1 — add missing[] + patterns[] fields (Bundle 4-pa…
artyhoo May 8, 2026
4751f8c
docs(phase-4): close Open #2 + log Bundle 4-partial-extended completion
artyhoo May 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/audit-self.yml
Original file line number Diff line number Diff line change
Expand Up @@ -265,3 +265,38 @@ jobs:
cd /tmp/fake-consumer-rn
bash "$GITHUB_WORKSPACE/packages/core/audit-self/audit-ai-docs.test.sh" || \
{ echo "::error::framework audits failed inside tmp consumer (react-next)"; exit 1; }

framework-self-detect:
name: Detector v1 — self-application snapshot (L1 invariant)
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: '20'
- name: Install workspace deps (hoists tsx + transitive semver)
run: npm install --silent
- name: Detect this repo and diff against frozen snapshot
run: |
ACTUAL=$(npm --prefix packages/core run detect --silent -- "$GITHUB_WORKSPACE")
EXPECTED=$(cat packages/core/detector/expected-self-detect.json)
if ! diff <(echo "$ACTUAL" | jq -S .) <(echo "$EXPECTED" | jq -S .); then
echo "::error::detector self-application snapshot drift — update packages/core/detector/expected-self-detect.json or fix the regression"
exit 1
fi
echo "✓ self-detect snapshot stable"
- name: Verify setup.sh --stack=$(detect) --dry-run idempotent
run: |
DETECTED=$(npm --prefix packages/core run detect --silent -- "$GITHUB_WORKSPACE" | jq -r .stack)
bash setup.sh --stack="$DETECTED" --dry-run --skip-aif-init --skip-deps > /tmp/run1.txt 2>&1
bash setup.sh --stack="$DETECTED" --dry-run --skip-aif-init --skip-deps > /tmp/run2.txt 2>&1
if ! diff /tmp/run1.txt /tmp/run2.txt > /dev/null; then
echo "::error::setup.sh --dry-run is not idempotent — L1 acceptance (a) violated"
diff /tmp/run1.txt /tmp/run2.txt
exit 1
fi
echo "✓ setup.sh --dry-run idempotent (L1 acceptance a)"
387 changes: 387 additions & 0 deletions docs/meta-factory/PHASE-4-PROMPT.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/meta-factory/aif-comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ EVALUATE runs checks через parallel `Task` agents, aggregate в weighted sc
| `/aif-rules-check` | `.ai-factory/RULES.md` flat list (downgrade projection из manifest) | Advisory (LLM judge) |
| `/aif-verify` | hooks/CI commands из manifest `check.type=command\|script` | Hard (toolchain) |
| `/aif-loop` rules | JSON conversion манифеста → AIF RULE-SCHEMA format | Structured executable |
| `/aif-fix`, `/aif-architecture`, etc. | `.ai-factory/skill-context/<skill>/SKILL.md` — stack overrides | Per-skill customization |
| `/aif-fix`, `/aif-architecture`, etc. | `.ai-factory/skill-context/<skill>/SKILL.md` — stack overrides | Per-skill customization — **✓ closed Phase 4** (commit `b5e16b7`, see [retros/phase-4.md](retros/phase-4.md) Reuse posture #4.6) |

Convergent rule format (§2 ↔ rules-manifest.json) делает touchpoint 3 **тривиально mapping'ся** — JSON-to-JSON конвертер, не семантический rewrite.

Expand Down
2 changes: 2 additions & 0 deletions docs/meta-factory/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ Layer 5 — Installer
}
```

> **v1.1 subset note (2026-05-08):** Detector ships `{stack, framework, runtime, confidence, severity, weight, source, rules, missing, patterns}` per Bundle 4-partial-extended (Phase 5 entry). Fields `language`, `structure`, `router` are v2 backlog. Authoritative current contract: `packages/core/detector/types.ts`. Layer 2 (Research Agent) consumes `missing` + `patterns`.

**Что НЕ делает.** Не принимает решений о правилах. Только наблюдение, факты, без интерпретации.

### 2.4 Layer 2 — Research Agent
Expand Down
150 changes: 150 additions & 0 deletions docs/meta-factory/phase-4-research.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Phase 4 — Step 0 entry research (Stack Detector v1)

> **Trigger:** [EXECUTION-PLAN.md §5.5](EXECUTION-PLAN.md) «Existing solutions research» — Phase 4 entry gate (forward Step 0 trigger documented in §5.5 retrofit block, post-Phase-3 retro 2026-05-08).
> **Method:** context7 MCP queries against `/lee-to/ai-factory`. Two adjacent libraries (`semver`, `package-manager-detector`) returned no JS-canonical match; flagged as «not in context7» per §5.5 fallback (no git clone fallback used).
> **Status:** transient artifact per §5.5 — ≤200 lines; may be archived once Phase 4 closes.
> **Question answered:** which Phase 4 capabilities are covered by AIF (or top alternatives) such that Phase 4 should reuse rather than build, and which remain unique value?

---

## §1. Capabilities Phase 4 will cover

Phase 4 (per [EXECUTION-PLAN.md:466-487](EXECUTION-PLAN.md)) extracts logic from `setup.sh:82-97` + `packages/core/detector-v0/detect-applicable-rules.ts` into `packages/core/detector/` v1, adding:

1. **Stack root detection** — framework + runtime from `package.json`, lockfiles, `tsconfig.json`, `next.config.*`
2. **Version-aware logic** — Next 15 vs 16, React 18 vs 19 ranges (semver-aware comparison)
3. **Confidence scoring** — `high | medium | low` per detected dimension
4. **CLI surface** — `npm run detect` emits structured JSON
5. **Snapshot testing** — detector output stable across runs (CI invariant)
6. **Self-application snapshot** — detector runs on root repo in CI; expected output frozen

Per §5.5: each capability requires existing-solution evaluation before drafting `PHASE-4-PROMPT.md`.

---

## §2. Tools resolved (context7)

| Tool | Library ID | Benchmark | Notes |
|---|---|---|---|
| AI Factory | `/lee-to/ai-factory` | 83.7 | Primary candidate per [aif-comparison.md §9](aif-comparison.md) forward implication |
| `semver` (npm) | **not in context7** | — | Resolves returned only `.NET`, Rust, Go, Python variants. npm `node-semver` has no `/org/project` ID surfaced. Flagged. |
| Package-manager / framework auto-detect | **not in context7** | — | Resolves returned Microsoft/Intel/Swift OS package managers — irrelevant. `antfu/package-manager-detector` not surfaced. Flagged. |

Local-clone fallback **not used** — npm `semver` is well-known stable API; package-manager-detector reuse is optional, not blocking. Both flagged as «no context7 source», proceeding with build decisions documented below.

---

## §3. Per-capability matrix

### 3.1 Stack root detection

**AIF detection is prompt-driven (LLM reads inline instructions in skill markdown), not callable code.** Cannot be invoked directly from a deterministic CLI/CI context. But AIF *artifacts* (`.ai-factory/DESCRIPTION.md`, `.ai-factory/skill-context/<skill>/SKILL.md`, `.ai-factory/ARCHITECTURE.md`) are structured files Phase 4 can **read** as primary source — that IS reuse, not just convergent design.

**Source priority (highest confidence first):**

| Priority | Source | Provenance | Confidence |
|---|---|---|---|
| 1 | `.ai-factory/DESCRIPTION.md` | Human-curated (per AIF docs) | `high` |
| 2 | `.ai-factory/ARCHITECTURE.md` | Human-curated, augments 1 | `high` |
| 3 | `.ai-factory/skill-context/<skill>/SKILL.md` | AIF-accumulated (via `/aif-evolve`) | `high` |
| 4 | `package.json` deps + lockfile signature | Heuristic — fallback when AIF absent | `medium` |
| 5 | `next.config.*` / `tsconfig.json` presence | Confirmation signal | `low` |

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF artifacts (priority 1-3) | **Read** `.ai-factory/*.md` files as source-of-truth | **Reuse via read-side** — no logic duplication, no LLM dependency. Extends [aif-comparison.md §5](aif-comparison.md) touchpoint 4 (originally write-only «meta-factory generates skill-context») with a new read-side flow not in the original 4-touchpoint matrix. |
| `packages/core/detector-v0/detect-applicable-rules.ts` (priority 4) | Manifest `requires-package` field, structured JSON | Already in repo — extends to fallback mode |
| `setup.sh:82-97` (legacy) | bash heuristic | Replaced by structured detector |

**Reuse posture:** detector becomes **deterministic bridge over AIF artifacts** with manifest-heuristic fallback when AIF not installed. Not a parallel implementation.

### 3.2 Version-aware logic (Next 15 vs 16)

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF | **None.** Detection is binary («Next.js detected: yes/no»). No major-version branching surfaced in queries. | Gap — AIF can't switch skill behavior on Next 15 vs 16 |
| npm `semver` | `semver.coerce(version)?.major`, `semver.satisfies(version, '>=15 <16')` | Standard library; battle-tested. **Verified 2026-05-08:** `semver@7.7.4` already transitive via `@typescript-eslint/rule-tester` — no explicit dep required. |
| Self-rolled regex | `/^[\^~>=<\s]*v?(\d+)\./` | ~10 LOC; fails on dist-tags (`canary`, `latest`), aliases (`npm:foo@1`), pre-release tags, complex `||` ranges |

**Verdict:** AIF lacks this — Phase 4 unique value. **Use `semver` (npm).** Already transitive (verified 2026-05-08 above) — no `package.json` edit needed; just `import semver from 'semver'` in `packages/core/detector/`. Self-rolled regex rejected — semver parsing is plumbing, not methodological contribution; edge cases (canary tags, RCs, aliases) inevitable in real fixtures.

### 3.3 Confidence scoring

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF `aif-loop` RULE-SCHEMA | `severity: fail \| warn \| info` × `weight: 2 \| 1 \| 0`; aggregate `score = sum(passed_weights) / sum(active_weights)` | **Weighted aggregate per phase** — maps cleanly to per-detection confidence |
| Phase 4 plan (PROPOSAL §8) | `high \| medium \| low` per detected dimension | Domain-specific naming |

**Convergent design:** AIF's `fail=2 / warn=1 / info=0` weight tiers are isomorphic to `high / medium / low`. Both express «how confident is this verdict». **Adopt naming alignment** so detector output can feed into AIF rules without reformatting (touchpoint 3 in [aif-comparison.md §5](aif-comparison.md)).

### 3.4 CLI surface

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF | Slash-command runtime (`/aif-fix`, `/aif-verify`) — invoked from inside an AI-coding session, not standalone CLI | Different UX paradigm: assumes Claude/Cursor/etc. as orchestrator |
| Phase 4 plan | `npm run detect` → JSON to stdout (also npm `bin: meta-factory detect`) | Standalone npm CLI — fits consumer install scenario |

**Verdict:** Different audiences. AIF CLI = AI-orchestrator-internal; ours = npm consumer's `package.json` scripts. **Build standalone CLI** — no reuse possible. Surface area is ≤30 LOC (yargs/commander overkill; plain `process.argv` parsing sufficient).

### 3.5 Snapshot testing

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF EVALUATE phase | Parallel Task agents run executable checks; results stored in `.ai-factory/evolution/<task>/` | Stateful, per-task artifacts; not snapshot-equality |
| Vitest `toMatchSnapshot` (already used in `packages/core/render/__snapshots__/`) | File-based snapshot equality; auto-update on `vitest -u` | Standard JS-side pattern, already in repo |

**Verdict:** Different paradigms (live execution vs static equality). **Build with vitest snapshots** — already proven in Phase 2/3 (`render-rules.test.ts.snap`). No AIF reuse.

### 3.6 Self-application snapshot in CI

| Solution | Mechanism | Differentiator |
|---|---|---|
| AIF `/aif-verify --strict` | Runs full toolchain on changed files; emits `aif-gate-result` JSON | Not framework-specific; doesn't pin expected detector output |
| Phase 4 plan | `audit-self.yml` job runs detector on root repo, diffs against frozen `expected-detect.snapshot.json` | Closes self-application invariant L1 (per [self-application.md](self-application.md) §2) |

**Convergent point (touchpoint 4 in [aif-comparison.md §5](aif-comparison.md)):** detector output → `.ai-factory/skill-context/<skill>/SKILL.md` for AIF runtime to consume. **First-class reuse opportunity** — Phase 4 emits skill-context overrides directly, making meta-factory feed AIF without separate adapter.

---

## §4. Reuse-vs-build decisions

| # | Capability | Decision | Rationale |
|---|---|---|---|
| 4.1 | Stack root detection | **Hybrid REUSE: read AIF artifacts as primary source, manifest heuristic as fallback.** | AIF inline detection is prompt-driven (not callable code), but AIF *artifacts* (`.ai-factory/DESCRIPTION.md`, `skill-context/*/SKILL.md`) are structured files we read. Detector = deterministic bridge over AIF, not parallel implementation. Source priority list in §3.1. |
| 4.2 | Version-aware logic | **Build with `semver` (npm) — already transitive.** | AIF gap confirmed. `semver@7.7.4` already in lockfile via `@typescript-eslint/rule-tester` (verified 2026-05-08); no explicit dep needed. Self-rolled regex rejected — parsing semver is plumbing, not methodological contribution. |
| 4.3 | Confidence scoring | **REUSE convergent — adopt AIF severity/weight schema.** | Adopt AIF RULE-SCHEMA semantics directly: emit `severity` + `weight` per AIF spec; derive human-friendly `confidence: high\|medium\|low` as a view. Reuse-of-design-decision — don't reinvent scoring tiers. |
| 4.4 | CLI surface | **Build standalone npm bin.** | Different audiences; AIF CLI not reusable. Keep ≤30 LOC. |
| 4.5 | Snapshot testing | **Build with vitest snapshots.** | Standard pattern, already in repo. AIF EVALUATE is different paradigm. |
| 4.6 | Self-application + AIF integration output | **REUSE skill-context format as output sink.** | Detector emits `.ai-factory/skill-context/{aif-fix,aif-implement,aif-architecture}/SKILL.md` — closes touchpoint 4 from aif-comparison.md §5 in Phase 4 instead of deferring to Phase 11. |

**Acceptance per §5.5:** ≥1 reuse decision required. Achieved via **3 reuses**: 4.1 (read AIF artifacts) + 4.3 (adopt AIF severity/weight schema) + 4.6 (write skill-context). Plus `semver` as upstream-stdlib reuse for 4.2.

**Net Phase 4 scope = 3 build + 3 reuse.** Detector positioning: deterministic bridge over AIF artifacts (read + write), not parallel implementation.

---

## §5. Verdict — proceed with PHASE-4-PROMPT.md draft

**GO. Phase 4 prompt to be drafted with these scope deltas vs initial EXECUTION-PLAN §466-487 description:**

1. **Add 4.1 read-side:** detector reads AIF artifacts (`.ai-factory/DESCRIPTION.md`, `ARCHITECTURE.md`, `skill-context/*/SKILL.md`) as priority 1-3 sources before falling back to manifest heuristic (priority 4-5). Source priority documented in detector schema. Estimated +0.5 day.
2. **Add 4.6 write-side:** detector emits skill-context overrides for top-3 AIF skills (`aif-fix`, `aif-implement`, `aif-architecture`). Pulls AIF integration touchpoint 4 from Phase 11 → Phase 4. Estimated +0.5-1 day.
3. **Confidence schema alignment:** detector output JSON includes both human label (`high|medium|low`) AND AIF-compatible `severity` + `weight`. Single emit, dual contract.
4. **Version-aware via `semver`:** **verified 2026-05-08** — `semver@7.7.4` already transitive via `@typescript-eslint/rule-tester`; no `package.json` edit needed.
5. **No revert of `detector-v0`:** v1 = extension of v0, not rewrite. Preserves Phase 3.1 commit history.

**Risks introduced by 4.1 + 4.6 AIF coupling:**
- AIF artifact format coupling on **both read and write sides** — if AIF changes `.ai-factory/` schema (per AIF coupling risk row in [risks.md](risks.md) 2026-05-08), detector breaks bidirectionally. Mitigation: schema validation in detector tests; subscribe AIF release notes; graceful degradation when AIF artifacts absent (fallback to priority 4-5 = manifest heuristic).
- Scope creep — pulling Phase 11 work into Phase 4 risks blowing 1-week budget. Stop-rule: if 4.1 read-side AND 4.6 write-side combined not done by day 5, ship 4.1 + defer 4.6 to Phase 11 with documented split-point. (4.1 read-side has higher priority — it's Phase 4 self-application angle; 4.6 write-side is bonus integration.)

---

## §6. Forward implications (watch-list)

| Item | Trigger | Owner |
|---|---|---|
| Migrate to `antfu/package-manager-detector` if/when it lands in context7 | next-phase-entry re-validation | Phase 5+ prompt author |
| AIF artifact schema validation (`DESCRIPTION.md`, `skill-context/*/SKILL.md`) — **mandatory in detector tests, not optional** (see §5 risk mitigation: bidirectional break on AIF schema change) | AIF v3.x release; also any 2.x minor that touches `.ai-factory/` shape | continuous (read + write side both affected) |
| Multi-stack monorepo detection (§13.5) | Phase 9+ entry | deferred |

These are watch-list items, not commitments. Re-validate via context7 at each Phase entry per §5.5.
Loading
Loading