Skip to content

docs(audit): refresh AUDIT-CHECKLIST/AUDIT-PROMPT post-C-1 + packages/ layout - #85

Merged
artyhoo merged 2 commits into
mainfrom
docs/audit-tooling-refresh
May 21, 2026
Merged

docs(audit): refresh AUDIT-CHECKLIST/AUDIT-PROMPT post-C-1 + packages/ layout#85
artyhoo merged 2 commits into
mainfrom
docs/audit-tooling-refresh

Conversation

@artyhoo

@artyhoo artyhoo commented May 21, 2026

Copy link
Copy Markdown
Owner

What

Refresh the recursive self-audit tooling (AUDIT-CHECKLIST.md, AUDIT-PROMPT.md) to current origin/main reality. Primary driver: the merged C-1 agent-collision resolution (docs/meta-factory/research-patches/2026-05-20-agent-collision-resolution.md, PRs #79/#82/#83/#84). Internal-only tooling — not shipped via install.sh, no consumer impact.

⚠️ Scope note (read first)

The kickoff framed this as C-1-only. While reworking I found the audit files predate the packages/ monorepo migration (last touched at bb8c316, a factory/ rename) — so they referenced factory/RULES.md, scripts/audit-ai-docs.sh, tests/audit/ etc. that had moved. The C-1 probes are entangled with those dead paths (probe 2.1 / Phase-2 item 9 grep factory/RULES.md and best-practices-sidecar.md), so I could not "run the reworked probes" (T2) or satisfy §4.3 ("run end-to-end, no stale-probe artifacts") without also fixing the layout paths.

I reconciled both — C-1 (headline) + layout — because it's all in the same two files / one deliverable, and §4.3 mandates a runnable end-to-end audit. If you wanted strictly C-1, say so and I'll split the layout part out.

Per-reference classification

Reference (old) Class Now points at / asserts
best-practices-sidecar.md "validates RULES.md" / "manual review" (§2.1, §3.2, Phase-2 item 9) REWORK enforcement strategy = audit-script header (delegated-ESLint / probe / manual, lines 8-18) + aif-rules-check skill-context residue (R4/R10/R17). KEEP-AIF: not ours.
best-practices-sidecar / review-sidecar / docs-auditor boundaries (§3.4, item 13) REWORK the 3 we actually ship: review-sidecar, living-docs-auditor, compliance-verifier.
.claude/agents/* "overridden by us" (§6.2) REWORK post-C-1 we occupy zero AIF slots; added comm -12 disjointness probe.
/aif-verify should invoke best-practices-sidecar (§6.3) REWORK invokes AIF rules-sidecar (reads RULES.md) + aif-review (our skill-context).
docs-auditor (ours) (§3.4, item 13, item 16) RENAME living-docs-auditor (1:1 map).
best-practices-sidecar.react.md "inlined" stale-path probe (§2.4, item 6) REMOVE target never existed in current repo (T-AUD-B); §2.4 now checks the real post-C-1 invariant (no old agent file / no active wiring).
factory/RULES*.md (§2.1, §3.2, item 7, item 10, item 21) REWORK packages/preset-next-15-canonical/RULES*.md
scripts/audit-ai-docs.sh as source path (§1.5, §2.2, item 8, item 21, item 23) REWORK packages/core/audit-self/audit-ai-docs.sh. (Kept as CONSUMER path in §5 / Phase-5 install tests — T16: same string, different intent.)
tests/audit/audit-ai-docs.test.sh (§8.2, item 22) REWORK packages/core/audit-self/audit-ai-docs.test.sh; "16 pass" → "9 pass".
factory/{DESCRIPTION,ARCHITECTURE}, templates/shared/tsconfig.json (§3.1, §3.5, item 15) REWORK packages/core/templates/shared/
templates/react-next/ (§2.5) REWORK packages/preset-next-15-canonical/templates/
templates/ts-server/ (§2.5, §3.5) KEEP still at repo root — deliberately kept per Phase-3 Gate 2 (option A), templates/shared+react-next deleted.
§8.1 "no CI for the package" gap REWORK RESOLVED — .github/workflows/audit-self.yml exists (14 jobs).
§8.2 "no negative tests" gap REWORK RESOLVED for ts-server (9 pass/0 fail); react-next still uncovered → kept as finding.

Run evidence (T2/T3 — every reworked/kept mechanical probe executed against current main)

1.1 git ls-files | wc -l                → 390 ✓
1.5 executables                         → FAIL: setup.sh not +x  (REAL FINDING, see below)
2.1 orphan-rule check (R1-R11)          → no ORPHAN ✓
2.2 R/D comment count                   → 5 (≥5) ✓
2.4 stale agent/path                    → empty ✓ (no file, no wiring)
2.5 templates ls                        → ts-server (root) + preset templates present ✓
3.5 node --check eslint + tsconfig json → clean ✓
8.1 CI jobs grep                        → mechanical/rule-to-probe/probe-tests +11 more ✓
8.2 / item22 negative-test run          → 9 pass / 0 fail ✓
item9/item21 rule-to-probe replication  → exit=0, no ORPHAN ✓
item23 self-audit run                   → exit 0; 4 PASS, 0 FAIL, 2 WARN (no AGENTS.md / no tool-decisions.md — acceptable) ✓

Sanity: markdownlint 0 errors (fixed 2 pre-existing MD040 bare-fence violations in the copy-paste prompt block → text); both files ≤500 (453 / 302); 21 embedded bash blocks, 0 bash -n errors. Pre-push: principles 81 pass, render-drift clean, self-tests 16 pass.

Genuine findings surfaced (NOT fixed here — audit-tooling refresh only, per kickoff §7)

  1. MINOR — setup.sh lacks the executable bit (git mode 100644; install.sh is 100755). Works via bash setup.sh, but inconsistent. §1.5 correctly FAILs on it (annotated as a known finding).
  2. MAJOR — no negative-test file for the react-next probes (packages/preset-next-15-canonical/audit-self/audit-ai-docs.react-next.sh: R17 + the R12/R14/R15/R16a/R16b/R20 ESLint-delegated probes). The ts-server probes are negative-tested; the react-next ones are not. §8.2 / item 22 flag it.

§1.7

§1.7 Skipped: internal audit-tooling doc refresh consequent to an already-merged decision (C-1); no new rule, principle, or discipline is introduced — and discipline-self-check does not fire on root *.md (path filter).

AI-laziness traps honored

  • T2/T3 — ran every reworked probe, pasted command output above; no prose-only "would pass".
  • T16 / T-AUD-A — did not sed-swap: scripts/audit-ai-docs.sh kept as the consumer path in install-test sections vs re-pointed to packages/core/audit-self/ in source-audit sections; templates/ts-server kept (Gate 2) while templates/shared/react-next moved. Each reference mapped by intent.
  • T-AUD-B — removed the dead best-practices-sidecar.react.md probe rather than inventing a new target.
  • T14 — the green is real (executed), not assumed.
  • T15 — these docs are the project's recursive self-audit; the refreshed audit run on current main is internally consistent (2 acceptable WARNs, 1 real finding correctly surfaced).

artyhoo added 2 commits May 21, 2026 03:03
…/ layout

Reconcile the recursive self-audit tooling to current origin/main reality.

C-1 agent-collision resolution (research-patches/2026-05-20-agent-collision-resolution.md):
- best-practices-sidecar → KEEP-AIF: no longer ours. R-rule enforcement strategy now
  documented in the audit-script header (delegated-to-ESLint / probe / manual) + the
  aif-rules-check skill-context residue (R4/R10/R17), not in a best-practices-sidecar.md.
  Reworked §2.1, §3.2, §3.4, §6.2, §6.3, Phase-2 item 9, item 13.
- docs-auditor → RENAME to living-docs-auditor throughout (§3.4, item 13, item 16).
- review via skill-context aif-review (§6.3); compliance-verifier ships unchanged.
- Removed the dead best-practices-sidecar.react.md stale-path probe (target gone); §2.4
  now checks the post-C-1 invariant (no old agent FILE / no active WIRING).

packages/ monorepo layout reconciliation (audit files predated the migration; required by
§4.3 "run end-to-end, no stale-probe artifacts" + T2 "run reworked probes"):
- factory/RULES*.md → packages/preset-next-15-canonical/RULES*.md
- scripts/audit-ai-docs.sh → packages/core/audit-self/audit-ai-docs.sh (source-repo path;
  scripts/audit-ai-docs.sh is the CONSUMER path, kept in Section 5/Phase-5 install tests)
- tests/audit/audit-ai-docs.test.sh → packages/core/audit-self/audit-ai-docs.test.sh
- factory/{DESCRIPTION,ARCHITECTURE} → packages/core/templates/shared/; templates/react-next
  → packages/preset-next-15-canonical/templates/; templates/ts-server kept at root (Gate 2).
- §8.1/§8.2 "known gaps" RESOLVED: audit-self.yml CI now exists (14 jobs); negative-test
  file runs 9 pass/0 fail. Updated expected-states (was "16 pass", was "no CI").

Every reworked/kept mechanical probe was run against current main and passes; two real
findings surfaced (NOT fixed here, per scope): setup.sh lacks +x (install.sh has it); no
negative-test file for the react-next probes.

Prior-art: skipped — audit-tooling doc refresh consequent to merged C-1, no new capability/dependency.
…anical CI

The audit-self.yml `mechanical` job greps the repo for the literal
`ai-factory-RULES.md`. The new historical-rename comment contained it verbatim;
write it as the escaped regex form `ai-factory-RULES\.md` (same convention as the
probe below it) so it documents the rename without tripping the stale-path check.

Prior-art: skipped — one-line CI fix on audit-tooling doc, no new capability/dependency.
@artyhoo
artyhoo merged commit 903826c into main May 21, 2026
17 checks passed
artyhoo added a commit that referenced this pull request May 21, 2026
…#106)

Append + in-place refinement of the living history book (v1 technical
+86 lines, v2 literary +78). Narrates post-#93 events: the end-of-turn
hook saga (#81 broken via systemMessage → #89 reason-channel fix),
#94 AskUserQuestion hook, principles 11/12/13 landing, the #96/#97
claim-scan experiment (H0 not rejected), #90 recommendation-gate, and
the niche-among-companions chapter. Colophon extended (chapters III–XII,
patches to 2026-05-21, PR #85#99). Preserves continuation work that was
stranded uncommitted on the superseded chore/ssot-karpathy branch.

Prior-art: skipped — narrative doc continuation, no new capability or dependency.
artyhoo added a commit that referenced this pull request May 21, 2026
…efore #101 landed (#108)

The niche-strategy roadmap (#101, merged 2026-05-21 18:09) listed N6a (C-1 impl)
and N4a (claim-detector fix) as pending/urgent, but both had merged earlier the
same day: N6a via #79+#82+#83/#84/#85 (incl. live-probe DECISION-NEEDED #2 passed),
N4a via #98 (detector v2, validated against committed eval baseline). Adds a
post-merge STATUS RECONCILIATION banner + inline DONE markers across N4/N6/§5,
without rewriting the original analysis. Remaining N4 work = N4b (gate) only.

Prior-art: skipped — docs status-reconciliation only, no new capability or dependency introduced.
@artyhoo
artyhoo deleted the docs/audit-tooling-refresh branch May 22, 2026 18:09
artyhoo added a commit that referenced this pull request May 29, 2026
…k + aif-handoff MCP) REFERENCE (#275)

Verdict per BFR-default §1: REFERENCE — design-vocabulary for dispatch-trigger
optimisation of Variant A; not actionable standalone.

Key findings (2026-05-29 evidence):
- CC PostToolUse hooks expose a purpose-built `mcp_tool` hook type for
  already-connected MCP servers (code.claude.com/docs/en/hooks.md fetched
  2026-05-29) — T-AIF-BRIDGE-B'-1 evades via different shape than originally
  framed.
- aif-handoff MCP server runs standalone without coordinator daemon for task
  creation (DeepWiki ×2 phrasings: «MCP server can be invoked statelessly from
  an arbitrary process» + «MCP HTTP daemon runs independently») —
  T-AIF-BRIDGE-B'-3 evades.
- 4 sub-options each fit <100 LOC bridge target: Option (a') = 0 LOC bash
  (mcp_tool hook config), Option (a) = ~25 LOC bash (command hook + curl),
  Option (b) = ~15 LOC chokidar Node, Option (c) = ~10 LOC fswatch bash.

BUT: B' inherits all 3 of SW-A's ADOPT-blockers verbatim (PLAN.md disk coupling,
WebSocket-broadcast no-topic-filter, autoMode Reviewer vs reviewer-discipline.md
§2 conflict — DN-1=B-constrained applies). B' optimises only the dispatch
trigger; the load-bearing blockers persist. Comparison table vs A and B in §6.

§4 5-criteria: PASS-PASS-PARTIAL-PASS-INHERITED. Match ~35%. Atomicity
INCONCLUSIVE/MITIGABLE (chokidar awaitWriteFinish for Option b; empirical for
hooks).

Falsifier: B' upgrades to ADOPT VOCABULARY iff Sub-wave D greenlights A AND
DN-1=B-constrained adopted.

T-discipline applied per kickoff §6 SW-B2: T1 / T3 / T7 / T11 / T12 / T13 / T15
/ T16 / T17 / T19 (own cold-QA before push) / T20 + T-AIF-BRIDGE-B'-1..5.

Prior-art: SSOT #20 (CC hooks API, ADOPT) + SSOT #27/#28/#30/#43/#44/#46/#67/#80
(aif-handoff rows). Additive note to SSOT #44 proposed (no verdict change).
Proposed new SSOT row at next-available slot #85.
artyhoo added a commit that referenced this pull request May 29, 2026
…verdict (DEFER-all-with-updated-triggers) (#281)

Cross-variant synthesis consolidating Sub-waves A/B/B'/C (merged PRs #267/#268/#275/#276). Per-variant verdicts re-derived independently per T15: A=REFERENCE (28%), B=REJECT (~5%), B'=REFERENCE conditional ADOPT VOCABULARY (~35%), C=REJECT (~22%). Cross-variant recommendation per kickoff §6.5: DEFER all variants with updated SSOT «Trigger to revisit» fields.

Includes 20-cell comparison table (4 variants × 5 criteria, file:line citation per cell), 4-row T16 problem-class match table, DN-1 (B-constrained) + DN-2 (B' is operative B-path) reflections, hook-discipline (Decision 10) analysis (no equipoise → recommend on clearly-better evidence; DECISION-NEEDED NOT warranted), consolidated SSOT additive notes for #27/#28/#30/#44/#67/#80, proposed new SSOT row #85 (next-after-#84 verified).

Prior-art: prior-art-evaluations.md#27, #28, #30, #44, #67, #80 (existing aif-handoff entries, additive-note-only — no verdict changes); SSOT row #85 PROPOSED, not landed.
artyhoo added a commit that referenced this pull request Jun 1, 2026
…p (Superset/Superpowers/AI-Factory/amux/CC-native)

Autonomous parallel Mode-A sweep (4 Opus workers) per maintainer 'пройтись по
всем спутникам'. OhMyOpencode + Cursor excluded per maintainer. §10 added +
SSOT #104-#108. Spot-check caught propagated error: '#85=amux' is wrong (#85 =
pre-commit); amux had no row -> #104 + correction. Top payload: CC-native
under-used features (/goal, orchestrator-gate hooks, /code-review --fix). CC
claims worker-sourced, flagged for bench-test before live wiring.

Prior-art: skipped — research patch §10 extension + SSOT append-only #104-#108, no new capability/code/dependency (T5 research-only)
artyhoo added a commit that referenced this pull request Jun 1, 2026
…on (Superset/aif/CC-native, agnostic lens) (#329)

* docs(research-patches): capability-census — full-population enumeration of Superset/aif-handoff/CC-native (agnostic lens)

Closes the T10 population-enumeration gap the survey (#321) + harvest (#328)
left open: each product's full capability population was never enumerated.
Two dual-channel-verified criterion-zero findings hitting live problems —
F1 CC paths: frontmatter <-> our inject-matching-rule.sh (ADAPT dual-channel,
'как с хуком'); F2 CC Routines /fire <-> broken runtime-bridge dispatch
(REFERENCE). SSOT rows #101-#103 appended (#100 reserved for #328).

Prior-art: skipped — research patch + SSOT append-only rows #101-#103, no new capability/code/dependency (T5 research-only)

* docs(research-patches): capability-census wave-2 — all-companion sweep (Superset/Superpowers/AI-Factory/amux/CC-native)

Autonomous parallel Mode-A sweep (4 Opus workers) per maintainer 'пройтись по
всем спутникам'. OhMyOpencode + Cursor excluded per maintainer. §10 added +
SSOT #104-#108. Spot-check caught propagated error: '#85=amux' is wrong (#85 =
pre-commit); amux had no row -> #104 + correction. Top payload: CC-native
under-used features (/goal, orchestrator-gate hooks, /code-review --fix). CC
claims worker-sourced, flagged for bench-test before live wiring.

Prior-art: skipped — research patch §10 extension + SSOT append-only #104-#108, no new capability/code/dependency (T5 research-only)
artyhoo added a commit that referenced this pull request Jul 17, 2026
…es + SSOT #102 fix (#1028)

* docs(meta-factory): archival audit of D.6 + capability-census R-patches + SSOT #102 fix

Dual-reviewer cycle (4 parallel agents: top-down + bottom-up on each of the
two older R-patches) surfaced three classes of staleness, all handled via the
archival convention (post-merge status note + body left intact; census-patch
§5 sets the 'R-patch is NOT rewritten' precedent).

D.6 R-patch (2026-06-02-deps-hash-check-dup.md): post-merge status note added.
  Verdict §5(b) was implemented in I-phase PR #383 (@dual-pair: deps-hash-check-
  dogfood marker + byte-identity test, both shipped, byte-identity actively
  enforced). IMPORTANT correction to §6: §6 prescribes ASYMMETRIC markers
  (File 1 dogfood = 4-line spec block, File 2 source = 1-line @dual-pair), but
  the I-phase kickoff deliberately overrode this — byte-identity is binding,
  so BOTH copies got the SAME marker block (asymmetric would have broken the
  byte-identity test). §6 is the R-phase proposal, NOT the shipped spec.
  Also flags stale line-refs (install.sh:261→421, copy_safe→refresh_safe,
  1754→3197 bytes, check-hook-marker.sh:42→54) — correct at 2026-06-02
  authoring, stale today. The D.6 verdict itself remains load-bearing — it is
  the origin of the @dual-pair invariant that deps-hash-multistack reuses.

capability-census R-patch (2026-06-01-capability-census.md): post-merge
  status note added. (a) F2 premise SUPERSEDED — §0/§3 F2/§10 assert
  'runtime-bridge dispatch currently broken', but #313 (REST dispatch) merged
  2026-05-31 20:14, ~9h BEFORE this census merged 2026-06-01 05:19; the census
  relied on a memory note, not staging. (b) Verdicts RESOLVED: F1 (#331+#593),
  #108 (#340). (c) #85=amux miscite still propagates in #328 (5 loci) + SSOT
  #100. (d) §10.3 self-contradiction: §2.1 is the CORRECTION site, not a
  propagation site (L44 carries 'NOT #85').

SSOT #102 (prior-art-evaluations.md): active row carried 'currently broken,
  memory runtime_bridge_mcp_dispatch_fix' — this is an ACTIVE SSOT row (not
  archival), so corrected in place to 'RESOLVED via #313 REST dispatch
  2026-05-31 — the «currently broken» framing is stale'. The census post-merge
  note cross-references this.

markdownlint 0 errors. diff = 3 files, 3/2.

Prior-art: skipped — archival docs audit + 1 SSOT row correction, no new
capability.

* chore: retrigger CI with corrected §1.7 PR body (file:line citations)

---------

Co-authored-by: t <t@t.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant