research(meta-orchestrator-stage-5): dogfood findings — substrate validated, 5 gaps surfaced - #245
Merged
Merged
Conversation
…idated mechanically, 5 gaps surfaced
Stage 5 dogfood for `meta-orchestrator-mode-triage-and-planner` umbrella per kickoff
`.claude/orchestrator-prompts/meta-orchestrator-mode-triage-and-planner/stage-5-dogfood-kickoff.md`.
Worker (Mode A inline Opus role=Worker autonomous, Option A canonical from kickoff §9.A)
exercised the no-arg `/meta-orchestrator` substrate end-to-end against current repo state.
Helpers exited 0 with output matching published schemas; SKILL.md §2.5 routing tree walked
line-by-line; Stage 4 `parse-override-flags.sh` parsed `--mode-solo --reason=…` correctly.
5 substrate gaps surfaced (no fixes applied per feedback_no_drive_by_prs):
- J1 MAJOR: classify-work.sh silent string-mode fallback on non-existent kickoff paths
poisons routing cascade for ~30 synthetic L1 candidates (false TYPE=fix).
- J2 MAJOR: priority-score.sh `for dir in PROMPTS_DIR/*/` lacks nullglob → phantom
`* kickoff=missing` row in empty/gitignored prompts dirs.
- J3 MINOR: kickoff §3 Step 2a uses `mapfile -t` (bash 4+); macOS bash 3.2 fails.
- J4 MINOR: kickoff §3 Step 4 `${CANDIDATE_IDS[0]:-}` empty under zsh (1-indexed arrays).
- J5 MAJOR: substrate broken in fresh worktrees because `.claude/orchestrator-prompts/`
is gitignored; affects priority-score real-kickoff surface, launch-table-generator,
classify-work, dup-detect, AND principle 12 sentinel (which fails 1<10 vs main repo).
N1+N2 (MINOR): kickoff `${CANDIDATE_IDS[0]:-}` ≠ SKILL.md §2 priority-scored winner;
sibling_count lacks deterministic helper.
T19 cold-review of own §1-§3 (findings §4): GO with one self-amendment (A6 folded into
J1 severity datum). No fabricated citations or prose-only claims detected.
Coherence verdict per N=1 framing (T-MMT-5-B): mechanically coherent (helpers exit 0
matching contracts), semantically misleading (cascade produces plausible-looking routing
on synthetic-id input that classify-work fabricates from path-string heuristic).
Stages 2A/2B/2C/3 (PRs #239-#243) + Stage 4 (PR #244) validated as built; this PR closes
the LAST step of meta-orchestrator-mode-triage-and-planner umbrella.
Prior-art: skipped — dogfood run, no new capability surface (per CLAUDE.md escape hatch).
…le 10 Pre-push hook surfaced principle 10 (research-patch-annotation) violation: findings doc lacked `<!-- scope:<slug> -->` on first line. Added per peer-patch convention (e.g. `2026-05-25-meta-orchestrator-mode-triage-and-planner-design.md` line 1). scope=meta-orchestrator-stage-5-dogfood
artyhoo
pushed a commit
that referenced
this pull request
Aug 10, 2026
…245) Merge-forward per .claude/rules/git-conflict-merge-forward.md (rebase / force-push is a dead end on a published branch). Sole conflict: this branch and #1362 both appended an SSOT row numbered 244. Resolution — staging's #1362 row KEEPS 244 (already merged, its own trailer cites that ID); this branch's GNU-make-resolver row is renumbered to 245 and ordered after it. Verified post-resolve: no conflict markers, no duplicate IDs across the table, sequence ends 243 · 244 · 245, markdownlint 0 errors, and the diff against staging is a single added line (the first resolve attempt also ate the blank line before the closing `---`, which turned the whole table into a setext heading — MD003 at :186; restored). CORRECTION to 4e4231a's trailer: it cites `prior-art-evaluations.md#244`, which after this merge is #1362's actionlint row. The correct citation for principle 36 is **#245**. Principle 11 F1 still resolves via the verbatim SSOT path match (row 245's "Capability matched" cell names packages/core/principles/36-make-target-claim-liveness.test.ts), and the squash body at merge time carries #245. Two staging commits landed mid-branch and each retires something this PR asserted: - #1362 wired `shipped-prettier` + `framework-fresh-install-validate-multistack` into `ci-success` `needs:` and gated it — exactly the GAP-FOUND in this PR's §1.7 Backward-check surface 2. Now CLOSED upstream; the PR body is updated so it does not go on asserting a defect that no longer exists. - #1363 sized the spawn-heavy hook suites to their real runtime — the local `test:hooks` flake this branch diagnosed (12 failures, all "timed out in 5000ms", all under hooks/; the same file passes 5/5 in 22.76s under --no-file-parallelism --testTimeout=30000). Note for the reviewer: #1362 independently BUILT a needs-completeness gate for the sibling surface while this PR built the make-target-claim gate. Two instances of one class — "an artifact asserts a falsifiable property about a channel it does not own" — were gated within an hour of each other, by different sessions, neither aware of the other. That is corroboration that the class is real, not a one-off. Prior-art: prior-art-evaluations.md#245 (GNU make `-n` as target-recipe resolver + the absence of any tool verifying a script's prose claim about which build target invokes it — HYBRID: ADOPT the resolver, BUILD only the claim-extraction slice).
artyhoo
pushed a commit
that referenced
this pull request
Aug 10, 2026
Third merge-forward on this branch; staging is moving fast today. #1368 appended its own row 245 while this branch held that number. Resolution is the same shape as the 244 → 245 one: the row already ON staging keeps its ID (its commit trailer cites it), and this branch's GNU-make-resolver row steps aside — now **246**. Verified post-resolve: no conflict markers, no duplicate IDs across the whole table, sequence ends 244 · 245 · 246, the blank line before the closing `---` is intact (its loss during the first resolve turned the table into a setext heading — MD003), and markdownlint reports 0 errors. Trailer bookkeeping: commits 4e4231a and bf5ef84 cite `#244` and `#245` respectively; after this merge the row is `#246`. Both citations are stale by renumber, not by substance — no verdict, rationale or evidence changed, only the row's position behind two upstream inserts. Principle 11 F1 resolves via the verbatim SSOT path match (row 246's "Capability matched" cell names packages/core/principles/37-make-target-claim-liveness.test.ts), which has been kept correct across every renumber, and the squash body at merge time carries #246. Prior-art: prior-art-evaluations.md#246 (GNU make `-n` as target-recipe resolver + the absence of any tool verifying a script's prose claim about which build target invokes it — HYBRID: ADOPT the resolver, BUILD only the claim-extraction slice).
7 tasks
artyhoo
pushed a commit
that referenced
this pull request
Aug 10, 2026
…sion with #1366) `gh pr list --state open` shows PR #1366 (branch claude/clever-kowalevski-975fb2, opened 2026-08-10T09:28:32Z) already adding `packages/core/principles/37-make-target-claim-liveness.test.ts`. It has the earlier claim on the slot, so this branch moves to 38 rather than making the two collide at merge. No registry or contiguity assertion exists over principle numbers (`ls packages/core/principles` + grep found none), so a gap would have been harmless anyway — but two files claiming one slot would not be. Renames the file and updates every reference: the test's own header, the discipline-self-check.yml and workflow-integrity.yml comments that name the enforcing gate, the run-local-ci-sweep.sh block header, and SSOT #245. Also drops a stale line in the discipline-self-check.yml header that still described the guard as `base=staging` after 9c00b25 changed it to `!= 'main'`. Trailer note: this is a rename, not a new capability — but the detector counts the new path as a ≥80-LOC add under `packages/`, and correctly refuses the escape hatch for it (the byte-identical carve-out does not apply, since the file's own header text changed 37→38). Citing the entry the artifact has carried since it was introduced is the accurate answer, not a bypass. Prior-art: prior-art-evaluations.md#245 (actionlint RuleGlob / zizmor / poseidon/wait-for-status-checks / blend/require-conditional-status-checks / GitHub-native required_status_checks API — verdict BUILD. Unchanged by this commit: the capability is the same required-context registrability gate, moved from slot 37 to slot 38.)
artyhoo
pushed a commit
that referenced
this pull request
Aug 10, 2026
…ger, and gate registrability Completes the half #1368 declared and deferred. That PR marked `discipline-self-check.yml`'s job `# required-context: no` and named the exact precondition in the marker: «To make it required, the path filter must first move from `on.pull_request.paths` into the job body … so the context always reports.» Operator ruling 2026-08-10 requires this context on staging, so the precondition had to be met. Met by the stronger route: the `paths:` filter is REMOVED, not reimplemented in the job body. §1.7 already has a declared out-of-scope mechanism — the `### §1.7 Skipped: <≥60 chars>` marker, which pull_request_template.md:61 emits as the DEFAULT for every PR — so a second scope list in the job body would have been a third hand-maintained copy of «what counts as discipline-bearing», in a PR whose whole subject is that such copies drift. Measured before removing it: of the 40 most recent merged base=staging PRs, 38 already carry a valid §1.7 section (11 Skipped markers, 27 Forward/Backward pairs); #1363 and #1337 are the only two that would newly redden, each needing the one template-default line. Also adds `if: base.ref != 'main'` — a promote PR aggregates ~90 squashed commits and has no single meaningful forward/backward pair, so today it gets a guaranteed RED. Deliberately not the siblings' `== 'staging'`: theirs is subject-scoped (a kickoff's fidelity verdict, a stale-base rebuild against staging), whereas §1.7 also applies to the base=epic/* PRs CLAUDE.md:121 permits agents to merge, and `== 'staging'` would silently drop that coverage. A job-level `if:` is safe where a trigger filter is not — «if a job is skipped due to a conditional, it reports success». Both derived lists gain the newly-declared context. The sweep's REGISTRATION STATE line is rewritten to name contexts instead of «the last three» / «the first two» — appending a sixth entry above it had silently falsified both phrases, which is the same positional-drift shape principle 37 exists to catch. The gate, arms (i)-(k). Principle 37's existing arms assert the declared set is restated consistently; none asserts a declared entry is safe to register at all. `hasPullRequestTrigger` checks only that a `pull_request:` trigger EXISTS, never that it is unfiltered — so flipping a marker to `yes` inside a `paths:`-filtered workflow passes every existing arm and then freezes every PR matching none of the filter. That is precisely what #1368 wrote in prose and deferred, and prose is not a mechanism (attention-is-not-a-mechanism.md §1). Arm (i) asserts it, (j) is the paired negative (re-add the removed filter → RED; also run on disk against the real tree, not only in-parser), (k) pins the parser's scope: `pull_request:` only (workflow-integrity.yml filters both push: and pull_request:), inline `on:` forms resolve to [] rather than a false «no trigger» RED, and job-level `if:` is never flagged. Added to principle 37 rather than shipped as a parallel principle: that file already owns the declaration model and its parsers, so a second principle would have had to duplicate them (`#sync-by-copy-paste`). Prior-art: prior-art-evaluations.md#247 (actionlint RuleGlob / zizmor / poseidon/wait-for-status-checks / blend/require-conditional-status-checks / GitHub-native required_status_checks API — verdict BUILD, narrow complement to #245. actionlint validates `paths:` glob SYNTAX but per DeepWiki on rhysd/actionlint, 2026-08-10, «does not analyze the implications for branch protection»; the marketplace answers to skipped-but-required are RUNTIME waiters that add a job to wait on other jobs, never a static assertion that a declaration is registrable.)
artyhoo
added a commit
that referenced
this pull request
Aug 10, 2026
…ger, and gate registrability (#1371) ## Summary Completes the half #1368 declared and explicitly deferred, and closes the gap that deferral left in the gate. #1368 (merged 10:26Z today) built the declaration model: every job declares `# required-context: yes|no` at the job, and principle 37 asserts both in-repo lists equal the declared set. It marked `discipline-self-check.yml`'s job `no`, naming the precondition in the marker itself: «To make it required, the path filter must first move from `on.pull_request.paths` into the job body … so the context always reports.» The 2026-08-10 operator ruling requires that context on `staging`, so the precondition had to be met. Two things here: the §1.7 context becomes registrable and is declared required, and principle 37 gains the arm that makes «declared required ⇒ actually reportable» a mechanism instead of a comment. **Scope note, stated plainly:** this branch was opened before #1368 existed (my in-flight probe ran at 09:33Z; #1368 was created at 09:39Z) and originally carried its own parallel list-gating design. That work is discarded — #1368's declare-at-the-job model is better, and this PR is rebased onto it. The net diff is 5 files. ## Changes **1. `discipline-self-check.yml` — registrable, and declared required.** The `paths:` trigger filter is removed, and the marker flips to `# required-context: yes`. GitHub's guidance is the reason the filter and the requirement are incompatible: «Workflows skipped due to path filtering, branch filtering, or commit messages will remain in a pending state and block merging. To avoid this, do not require workflows that can be skipped.» Removed rather than reimplemented in the job body. §1.7 already has a declared out-of-scope mechanism — the `### §1.7 Skipped: <≥60 chars>` marker, which `.github/pull_request_template.md:61` emits as the DEFAULT for every PR. A scope list inside the job body would have been a third hand-maintained copy of «what counts as discipline-bearing», in a PR whose subject is that such copies drift. **Measured before removing it:** of the 40 most recent merged base=staging PRs, 38 already carry a valid §1.7 section (11 Skipped markers, 27 Forward/Backward pairs). #1363 and #1337 are the only two that would newly redden, each needing the one template-default line. Also adds `if: base.ref != 'main'`: a promote PR aggregates ~90 squashed commits and has no single meaningful forward/backward pair, so today it gets a guaranteed RED. Deliberately not the siblings' `== 'staging'` — theirs is subject-scoped (a kickoff's fidelity verdict, a stale-base rebuild against staging), whereas §1.7 also applies to the base=`epic/*` PRs `CLAUDE.md:121` permits agents to merge, and `== 'staging'` would have silently dropped that coverage. Safe because a job skipped by `if:` reports `skipped`, which protection accepts — «if a job is skipped due to a conditional, it reports success». **2. Both derived lists gain the newly-declared context**, per #1368's rule that the three move together. The sweep's `REGISTRATION STATE` line is also rewritten to name contexts instead of «the last three» / «the first two»: appending a sixth entry above it silently falsified both phrases — the same positional-drift shape principle 37 exists to catch. **3. Principle 37 arms (i)–(k) — registrability.** The existing arms assert the declared set is *restated consistently*. None asserts a declared entry is *safe to register at all*: `hasPullRequestTrigger` (`:112`) checks only that a `pull_request:` trigger EXISTS, never that it is unfiltered. So flipping any marker to `yes` inside a `paths:`-filtered workflow passes every existing arm and then freezes every PR matching none of the filter. That is exactly the hazard #1368 wrote into the `discipline-self-check.yml` marker and deferred — and prose is not a mechanism (`.claude/rules/attention-is-not-a-mechanism.md:1`). - **(i)** every `required-context: yes` job sits behind an unfiltered `pull_request:` trigger (no trigger at all is equally an offender). - **(j)** paired negative — re-adding the removed filter must be detected. - **(k)** parser scope: `pull_request:` only (`workflow-integrity.yml` filters *both* `push:` and `pull_request:`, so a file-wide scan would look identical here); inline `on: [push, pull_request]` resolves to `[]` rather than a false «no trigger» RED; job-level `if:` is never flagged, because flagging it would forbid the promote-flow guards all three PR-body gates rely on. Added to principle 37 rather than shipped as a parallel principle: that file already owns the declaration model and its parsers, so a second principle would have had to duplicate them. ## Operator action required **Order matters — register only AFTER this merges.** `discipline-self-check.yml` on `staging` is still `paths:`-filtered until then; registering its context first deadlocks every non-matching PR. Live state re-read 2026-08-10: `{"strict":false,"checks":[{"app_id":15368,"context":"ci-success"},{"app_id":15368,"context":"fidelity-verdict-in-pr-body"},{"app_id":15368,"context":"stale-revert-in-pr-diff"}]}` — three registered against six now declared. ```bash gh api -X PATCH repos/artyhoo/getff/branches/staging/protection/required_status_checks --input - <<'JSON' { "strict": false, "checks": [ {"context": "Template render probes — P1/P4/P6 (deterministic)", "app_id": 15368}, {"context": "capability PR carries Prior-art line in PR body (squash-survival)", "app_id": 15368}, {"context": "ci-success", "app_id": 15368}, {"context": "fidelity-verdict-in-pr-body", "app_id": 15368}, {"context": "stale-revert-in-pr-diff", "app_id": 15368}, {"context": "§1.7 forward+backward sections present in PR description", "app_id": 15368} ] } JSON ``` - Payload generated from the declared set, not retyped; `strict: false` + `app_id: 15368` reproduce the current pinning. Passing bare `contexts[]=` would drop the app pinning. - This registers all six declared contexts, including the two #1368 left declared-but-unregistered. Register a subset if you want them staged separately — but every unregistered declared context is one that can go red while the PR merges. - **PRs open at registration time will not report the new checks until they get a fresh PR event** (a push or a body edit — all the relevant workflows carry `types: [… edited …]`). - Verify: `gh api repos/artyhoo/getff/branches/staging/protection --jq '.required_status_checks.contexts[]'` → 6 lines. **Effect on the 6 currently-open PRs, measured (`gh pr list --state open`):** #1366 carries a valid §1.7 section. The other five — #1331 plus four Dependabot bumps (#1230, #1217, #1216, #1149) — do not, and will show a red §1.7 once this merges. This does **not** create a new blocked class: those four are *already* blocked by an existing required check — `gh pr checks 1230` shows `fidelity-verdict-in-pr-body fail` today, because Dependabot's body carries no `## Fidelity verdict` either, and Dependabot cannot edit its own body. Whatever already handles that for these PRs handles this too. Worth an explicit decision eventually; out of scope here. ## Test plan - [x] `npx vitest run packages/core/principles/37-required-context-completeness.test.ts` → 11/11 (8 pre-existing + 3 new). - [x] **Paired-negative on disk, not just in-parser.** The `paths:` filter was re-added to the real `discipline-self-check.yml` and the suite re-run: arms (i) and (j) both RED, with the deadlock message naming the job. Restored → 11/11 green. - [x] `npm --prefix packages/core run test:principles` → 40 files, 381 passed, 1 skipped, 0 failed. - [x] `bash scripts/run-local-ci-sweep.test.sh` → ALL PASS (the sweep's own paired-negative, after editing its header block). - [x] `actionlint` on both changed workflows → clean; `bash -n scripts/run-local-ci-sweep.sh` → clean; `npm run format:check` → clean. - [x] Full `bash scripts/run-local-ci-sweep.sh` on the pre-merge tree: PASS through `vitest-composition`; `format-check`, `rule-index-check`, `alwayson-budget`, `test:render`, `test:ir` also green individually. - [x] Live branch protection re-read before writing the operator command. **Pre-existing failures on this host, NOT introduced here** (both reproduced on a clean `staging` checkout, so out of scope per CLAUDE.md «PR strategy»): - `packages/core/hooks/getff-work.test.ts` — 3 failures (`CC-DEFERRAL`, `NON-CC-PRINT`, `NO-LAUNCH-FLAG`, all `expected 1 to be +0`). Verified identical on `/Users/art/code/rules-as-tests-aif` at `staging`. - `packages/core/principles/20-bundle-classification.test.ts` — one 30 s timeout under full-suite parallel load; passes in isolation in 45 s (~3.3 s/test), and green in the 40-file run above. Same spawn-heavy class #1363 addressed. ## Provenance n/a — not a stage PR. Direct implementation of the 2026-08-10 operator ruling, rebased onto #1368 after that PR merged mid-flight. ## Review findings n/a — not a stage PR. Three findings came from this branch's own pre-handoff self-review and were fixed before handoff: the `!= 'main'` guard scoping (`== 'staging'` would have dropped epic/* coverage); the inline `on: [push, pull_request]` grammar in the arm-(i) parser (an unhandled inline form would have false-REDded the one unconditionally safe shape); and the sweep's positional «last three» claim, which this PR's own sixth list entry falsified. **Also surfaced, not fixed** (no drive-by, per CLAUDE.md «PR strategy»): `origin/staging` currently carries **two** principle-37 files — `37-make-target-claim-liveness.test.ts` (#1366) and `37-required-context-completeness.test.ts` (#1368) — merged within minutes of each other. Nothing asserts principle-slot uniqueness, which is the same missing-mechanism class as everything else in this PR. ## Fidelity verdict FIDELITY: skipped — no kickoff or spec governs this change; it implements a direct operator ruling recorded in the task brief, not a stage of a planned umbrella. ## Parked questions n/a — not a stage PR. ## §1.7 Self-discipline check (REQUIRED if PR touches discipline-bearing files) ### §1.7 Forward-check applied **`no-paid-llm-in-ci.md §2`** — arms (i)–(k) are `readFileSync` + regex, zero API-billed calls; the workflow edits remove a trigger filter and add no step (`packages/core/principles/37-required-context-completeness.test.ts:71` imports only `node:fs`/`node:path`/`node:url`). **`ci-tool-pinning.md §1`** — no bare `run:` tool install added or altered; `.github/workflows/discipline-self-check.yml:79` still has no install step. **`attention-is-not-a-mechanism.md §1`** — this is the rule's own case: the deferred fix lived as a prose marker comment on a job, i.e. detection resting on someone reading it; arm (i) replaces it with a deterministic gate, and the surviving best-effort job at `workflow-integrity.yml:32` stays explicitly labelled non-load-bearing. **`rule-enforcement-channel-selection.md §3`** — mechanically detectable → gate; a principle test is the earliest firing channel (`packages/core/hooks/pre-push.ts:1269` runs `test:principles` at pre-push, CI as backstop). **`build-first-reuse-default.md §3`** — DeepWiki `rhysd/actionlint` + context7 `/github/docs` ×3 phrasings + WebSearch ×2 ran before the BUILD verdict; recorded as SSOT #247 in `docs/meta-factory/prior-art-evaluations.md:319` and cited by the commit trailer. The same rule drove the *shape*: the arms were folded into the existing principle 37 rather than shipped as a parallel principle, reusing its declaration model and parsers instead of duplicating them. **`language-discipline.md §1`** — all added comments, test prose and commit bodies are English. **CLAUDE.md capability-commit gate** — the diff adds no new file ≥80 LOC and no dependency; the trailer cites the SSOT entry anyway, since the capability is real even though the artifact is an extension. ### §1.7 Backward-check applied Delegated cold to `agents/backward-sweep-auditor.md` per T21 — handed only the two class predicates, never the diff, branch or PR, so it could not restate this change. (Dispatched against the pre-merge tree; both classes are structural and unaffected by the rebase, and finding B-2 below is about files this PR does touch, so it was re-checked by hand against the merged state.) **Class A — «a hand-maintained list whose correct contents are determined by a population declared elsewhere, with nothing asserting the mirror».** Population 13, all reached. SWEPT-CLEAN: `packages/core/principles/27-*` (install.sh copy-list ⊇ import graph), `.claude/rules/00-rule-index.md` + AGENTS.md (`scripts/render-rule-index.mjs --check`, `packages/core/hooks/pre-push.ts:1253`), agents/rules/skills header coverage (`packages/core/principles/09-doc-authority-hierarchy.ts:228`), `tests/agnosticism` probes (`run-audit.sh:14` globs, no hand list), `.claude/hooks/**` channel markers. **Six GAP-FOUND, none in this diff:** (1) the Tier-0 core set exists in three copies — `packages/core/principles/31-rule-channel-declaration.ts:58` (3 entries) vs `scripts/render-rule-index.mjs:56` and `scripts/render-rule-channels.mjs:75` (2 each), the latter's comment at `:70` claiming a «verbatim» mirror while differing, and `--check` regenerates *from* the stale copy so it stays green; (2) `packages/core/principles/09-doc-authority-hierarchy.ts:28` `REQUIRED_HEADER_DOCS` is static, and 5 `docs/meta-factory/*.md` carry no header; (3) `tests/install-sh/meta-all-wired.test.sh:23` loops only `tests/install-sh/*.test.sh`, leaving 29 of 127 tracked `*.test.sh` unasserted, 13 invoked by nothing; (4) `setup.d/LAYERS.md:3` declares itself authoritative for the layer list but carries 13 rows against 16 globbed by `install.sh:1142`; (5) `setup.d/10-skills.sh:108` names 12 of 14 skills, in sync but unasserted; (6) `scripts/run-local-ci-sweep.sh:123` gate table (low severity — a stale row yields a false LOCAL green and gates no merge). **Class B — «a declared gate whose channel cannot fire where it is relied upon».** Population 9, all reached. SWEPT-CLEAN: `guard-liveness-fullsweep.yml:20`, `pr-body-prior-art.yml:28`, `audit-self.yml:549` lychee vacuity guard, `audit-self.yml:1058` toolchain non-vacuity, `scripts/host-verify-coverage.sh:4`. `workflow-integrity.yml`'s own `paths:` filter is clean precisely because it is declared `required-context: no` — which arm (i) now asserts rather than assumes. **Three GAP-FOUND:** (1) 113 of 262 `packages/core` test files execute at no channel — `packages/core/vitest.config.ts:9` allowlists 22 dirs, pre-push runs 4 `test:*` scripts and CI 7, so `validator/gate-conflict.test.ts` and six other self-described gates never run; `audit-self.yml:389` states the class in the repo's own words and `:394` defers it; (2) all three PR-body required gates skip on base=`epic/*` while `CLAUDE.md:121` permits agents to merge there; (3) changed-markdown link integrity is pre-push-only and degrades to a warning when lychee is absent (`packages/core/hooks/pre-push.ts:1499`), while `link-checker.yml:26` is paths-filtered to `lychee.toml`. **Acted on within scope:** B-2 is the one finding this PR's own files carry, and it is fixed here for the §1.7 gate — `discipline-self-check.yml:77` uses `!= 'main'`, so base=`epic/*` stays covered. The cold agent reached that defect independently, on the two sibling gates, without seeing the diff; the two siblings keep `== 'staging'` and are **not** changed here, since narrowing them is a different subject with a recorded prior decision (`docs/superpowers/specs/2026-07-23-acceptance-contour-design.md:161`). The other eight findings are sibling surfaces outside this umbrella and are surfaced, not fixed — the Tier-0 three-copy set is the closest structural twin of what principles 36 and 37 each fixed once. Prior-art: prior-art-evaluations.md#247 (actionlint RuleGlob / zizmor / poseidon/wait-for-status-checks / blend/require-conditional-status-checks / GitHub-native required_status_checks API — verdict BUILD, narrow complement to #245. actionlint validates `paths:` glob SYNTAX but per DeepWiki on `rhysd/actionlint`, 2026-08-10, «does not analyze the implications for branch protection» and «does not have a feature to cross-check a list of status-check context strings against job `name:` fields»; the marketplace answers to skipped-but-required are RUNTIME waiters that add a job to wait on other jobs, never a static assertion that a declaration is registrable.)
artyhoo
added a commit
that referenced
this pull request
Sep 6, 2026
…quiring a resolvable referent (#1597 ledger L-1/B-3, K-5) (#1652) ## Summary Two ways the capability gate disagreed with the definition it claims to mirror — ledger #1597 findings **L-1 / B-3** (the wide half, == C13 addendum B-3) and **K-5** (the real half). `CLAUDE.md:30` declares the prose definition a mirror of `packages/core/hooks/checks/prior-art.ts`; both halves of the disagreement are closed here, in the same commit as the prose. ## Changes - **L-1 / B-3 — the LOC triggers taxed test material.** `packages/core/hooks/checks/prior-art.ts:231` and `:249` now skip test material in both LOC arms (`isExemptTestMaterial`, `packages/core/hooks/checks/prior-art.ts:221`): `*.test.*` / `*.spec.*` files and anything under `test(s)/`, `__tests__/` or `*fixtures/`. The carve-out cannot hide a capability — a commit that adds test material **alongside** a qualifying production file still trips on the production file — and it stops at `packages/core/principles/` (`ENFORCEMENT_FILE_RE`, `packages/core/hooks/checks/prior-art.ts:190`): a principle IS the enforcement capability, not a test for one. - **K-5 — a positive trailer needed no referent.** `checkTrailerBody` accepted any ≥20-char payload that was not the `skipped` escape hatch, so `Prior-art: consulted — no entry applies` satisfied the gate on a capability commit. A positive line must now name something a reader can open (`REFERENT_RE`, `packages/core/hooks/checks/prior-art.ts:74`): an SSOT row (`prior-art-evaluations.md#N`), a concrete artefact path, or an issue/PR reference. Stacked lines keep working — a referent-free line is skipped so a later valid line can carry the commit. - **Prose parity in the same commit.** `CLAUDE.md:36` now names three carve-outs on the LOC triggers instead of two, and `CLAUDE.md:50` enumerates the three referent forms. Three prose↔hook sync arms pin it (`packages/core/hooks/checks/prior-art.test.ts:1462`). - **Diagnostics.** `packages/core/hooks/pre-push.ts:383` names the three accepted referent forms in the failure message. ## Measurement (why each side moved) **L-1 / B-3 — the detector is the drifted side.** Over the last 250 first-parent commits on staging the ≥80-LOC arm fired on **27** commits; **18** of them added ONLY test files (9 under `packages/core/principles/`, 4 hook tests, 3 `packages/runtime-bridge/test/`, 2 skills, 1 synthesizer). The trailers it forced on the non-principle ones cited rows the commit never touched — `#242/#20/#16` on a hook-test pair (the C15 case), `#45 «unchanged by this»` on a runtime-bridge pair. The 9 principle files, by contrast, each carried a substantive on-topic citation (#244 actionlint, #245 safe-settings, #246 Vitest `test.include`, #251 markdownlint, #19 lychee, #229, #230, #248) — which is why the carve-out stops there. The split matches the **other enforcement channel of the same invariant**, which has held this exact semantic since it shipped: principle 11 (SSOT #48) builds its capability set from non-test files only (`packages/core/principles/11-build-first-reuse-default.test.ts:192`) while singling principle tests out as needing «a dedicated SSOT entry with verbatim path OR a Prior-art trailer» (`packages/core/principles/11-build-first-reuse-default.test.ts:525`). This PR brings the pre-push channel into parity with the CI one. **K-5 — error, not a transitional window, and a grammar wider than the strict reading.** Measured over the post-cutoff (`≥2026-05-12`) first-parent history: **145** capability commits carry a positive `Prior-art:` trailer. | grammar | commits it would reject | rejects `consulted — no entry applies`? | |---|---|---| | strict `#ID` only (the literal documented form) | **23 / 145 (16%)** | yes | | SSOT row \| artefact path \| issue/PR ref (shipped) | **2 / 145 (1.4%)** | yes | The 23 the strict reading would reject are legitimate consults naming in-repo precedent or a research patch (`setup.d/lib.sh:359`, `research-patches/2026-05-23-guard-liveness-gate.md §2`, `PR #261`), so the strict reading was rejected on evidence. The 2 that still fail under the shipped grammar are both from before 2026-07-19. No transitional window: the pre-push arm only inspects unpushed commits, so merged history is never re-checked, and the recent rate is 1 in the last 250 first-parent commits. ## Review findings - MAJOR: the ≥80-LOC / ≥50-LOC triggers classed new test files as capability commits, so two thirds of the gate's firings demanded a build-vs-reuse consult for work that adds no capability, and got rote citations back. Failure-scenario: a commit whose only new packages/ file is a 100-line `packages/core/hooks/hook-emit-prelude.test.ts` covering an existing prelude — `detectCapabilityReason` returned `"new file ≥80 LOC under packages/"` against the pre-fix module (probe output quoted under Test plan) and the real-git repro `tests/hooks/prior-art-trailer-hook.test.sh` sub-test 11 exited non-zero before the fix; both are clean after it, while sub-test 12 (a new principle file) stays non-zero on both sides. - MAJOR: a positive `Prior-art:` trailer that named nothing satisfied the gate on a capability commit. Failure-scenario: a commit adding the explicit dependency `some-new-dep: ^1.0.0` with the body line `Prior-art: consulted — no entry applies` — `checkTrailerBody(...).code === 0` against the pre-fix module and the real-git repro sub-test 13 exited 0, so a genuinely new dependency merged with a trailer carrying no referent to any register row, artefact, or PR; the squash message then propagated it as the project's own precedent for what a consult looks like. ### §1.7 Forward-check applied Both fixes land on the channels the gate already owns — the pre-push §7 arm (`packages/core/hooks/pre-push.ts:367` runs the same detector) and the PR-body arm (`packages/core/hooks/checks/pr-body-prior-art-bin.ts:23`) — with no new channel introduced, per the earliest-reachable-channel invariant. The prose↔hook obligation that `CLAUDE.md:30` asserts is itself mechanised rather than left to attention: `packages/core/hooks/checks/prior-art.test.ts:1462` reads `CLAUDE.md` from disk and fails if the capability definition stops naming the test-material carve-out or the `packages/core/principles/` exception, and if the trailer-syntax section stops naming all three referent forms — the counter to `#warning-nobody-reads` in `.claude/rules/attention-is-not-a-mechanism.md:29`. Each sync arm ships with a paired negative (`packages/core/hooks/checks/prior-art.test.ts:1476`). ### §1.7 Backward-check applied Swept the sibling surfaces of both classes. The LOC arms had exactly two call sites, both converted (`packages/core/hooks/checks/prior-art.ts:231`, `packages/core/hooks/checks/prior-art.ts:249`); the third detector arm (`isNewDepAdded`) is path-independent and unaffected. The parallel enforcement channel for the same invariant was checked before choosing the carve-out shape and already excluded test files (`packages/core/principles/11-build-first-reuse-default.test.ts:192`), so this narrows a divergence rather than creating one. On the trailer side, the three existing tests that encoded the pre-K-5 grammar were found and updated in place rather than deleted — the two space-stripping mutation-killers keep their exact length arithmetic with `#1271` standing in for filler (`packages/core/hooks/checks/prior-art.test.ts:577`, `packages/core/hooks/checks/prior-art.test.ts:1051`), and the C1 free-form case now asserts the honest post-fix semantic: a non-SSOT referent passes the existence arm because it has nothing to resolve (`packages/core/hooks/checks/prior-art.test.ts:807`). The two sibling PR-body gates (`packages/core/hooks/checks/pr-body-fidelity.ts`, `packages/core/hooks/checks/pr-stale-revert.ts`) parse their own trailers and share no code with this grammar. No static import was added from this shipped check to an unshipped util — the change is two module-local regexes. ## Fidelity verdict FIDELITY: skipped — no kickoff or spec substrate; this is a direct ledger-driven fix to the prior-art gate, verified by paired negatives that are RED against the pre-fix module and by a measurement over the repository's own first-parent history. ## Test plan - `npx vitest run hooks/` (in `packages/core`) — **1399 passed / 73 files**, including the 26 new arms in `packages/core/hooks/checks/prior-art.test.ts` (test-material carve-out across 9 path shapes, four paired negatives, the referent grammar with its vacuity probes and stacked-line cases, three prose↔hook sync arms). - `npx vitest run principles/ --no-file-parallelism` (in `packages/core`) — **477 passed / 2 skipped / 47 files**. - `bash tests/hooks/prior-art-trailer-hook.test.sh` — **19 pass / 0 fail** after the fix (4 new real-git sub-tests). Before it: **17 pass / 2 fail**, the failures being exactly new sub-tests 11 and 13; sub-tests 12 and 14 (the paired negative and the paired positive) are green on both sides, so the carve-out is narrowed and the grammar widened only where intended. - RED evidence against the pre-fix module (`git show HEAD:packages/core/hooks/checks/prior-art.ts` copy, run under tsx): - `detectCapabilityReason(test-only commit)` = `"new file ≥80 LOC under packages/"` → `null` after - `detectCapabilityReason(principles/99-new.test.ts)` = `"new file ≥80 LOC under packages/"` on **both** sides (control) - `detectCapabilityReason(hooks/checks/newcheck.ts)` = `"new file ≥80 LOC under packages/"` on **both** sides (control) - `checkTrailerBody('Prior-art: consulted — no entry applies').code` = `0` → `1` after - the SSOT-row, artefact-path and PR-reference trailers = `0` on **both** sides (controls) - `npx tsc --noEmit -p packages/core` — clean. - `bash scripts/build-getff-dist.sh --check` — «in sync with the repo root (1064 files)»; MANIFEST regenerated in the same commit as the files it pins, and re-verified after the merge-forward. - `SNAPSHOT_MODE=capture bash tests/install-sh/snapshot.sh` — three cells move in each of the eight npm baselines (`packages/core/hooks/checks/prior-art.ts`, `packages/core/hooks/pre-push.ts`, `.ai-factory/refresh-baseline.json`); cargo / python / go untouched, no host-drift hunk to discard this time. Re-run after the merge-forward: no further change. - Pre-push hook green on the pushed head. ## Prior-art consult Prior-art: prior-art-evaluations.md#56 (§7 Prior-art trailer parser — capability-commit detection + trailer validation, verdict BUILD; this change is a semantics fix inside that row's own module, no new capability and no new dependency). Prior-art: prior-art-evaluations.md#23 (commitlint, verdict ADOPT VOCABULARY — «custom trailer enforcement via regex on raw commit body»; the referent grammar stays inside that adopted vocabulary, adding no dependency, and the row's revisit trigger «§9 hand-roll grows beyond 2 substance arms» is unchanged: this is the §7 arm, not §9). Not a capability change: every touched path is a modification — no new file under `packages/`, no new explicit dependency.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage 5 dogfood for
meta-orchestrator-mode-triage-and-plannerumbrella. Worker (Mode A inline Opus, Option A canonical from kickoff §9.A) exercised the no-arg/meta-orchestratorsubstrate end-to-end against current repo state. Helpers exited 0 matching published schemas; SKILL.md §2.5 routing walked line-by-line; Stage 4parse-override-flags.shparsed--mode-solo --reason=…correctly. 5 substrate gaps surfaced + 2 minor design notes — all observation-only per feedback_no_drive_by_prs, no fixes applied.Single deliverable: docs/meta-factory/research-patches/2026-05-26-meta-orchestrator-stage-5-dogfood.md (367 lines).
Findings preview (§5 of patch — full repros inside)
classify-work.shsilent string-mode fallback on non-existent kickoff paths poisons routing cascade for synthetic L1 candidates (falseTYPE=fix)..claude/skills/meta-orchestrator/helpers/classify-work.sh:25-44priority-score.shfor dir in PROMPTS_DIR/*/lacksnullglob→ phantom* kickoff=missingrow in empty/gitignored prompts dirs..claude/skills/meta-orchestrator/helpers/priority-score.sh:62mapfile -t(bash 4+); macOS bash 3.2 fails..claude/orchestrator-prompts/meta-orchestrator-mode-triage-and-planner/stage-5-dogfood-kickoff.md:99${CANDIDATE_IDS[0]:-}empty under zsh (1-indexed arrays)..claude/orchestrator-prompts/meta-orchestrator-mode-triage-and-planner/stage-5-dogfood-kickoff.md:126.claude/orchestrator-prompts/gitignored): affects priority-score real-kickoff surface, launch-table-generator, classify-work, dup-detect, AND principle 12 sentinel (failed 1<10 vs main repo on first push). Worker workaround: copy main-repo prompts dirs into worktree (substrate-write-to-gitignored is expected per SKILL.md §10).${CANDIDATE_IDS[0]:-}≠ SKILL.md §2 priority-scored winner (alphabetical-first ≠ multi-criteria judgment).sibling_countlacks deterministic helper (Worker-judgment per current spec).Coherence verdict (N=1 framed per T-MMT-5-B)
Mechanically coherent (helpers exit 0 matching contracts, routing tree walked per SKILL.md §2.5 line 197-209). Semantically misleading on this invocation: cascade produced plausible-looking
Mode=DIRECT/direct-Editon a synthetic-id input thatclassify-work.shfabricatedLOC=1, SURFACES=1 → fixfrom the path-string heuristic, not from real kickoff analysis. Substrate works on THIS top-item-from-this-invocation; does NOT generalise to «works on synthetic discovery items».T19 cold-review verdict
GO with one self-amendment (§4 A6 folded into J1's reproduction notes — sibling_count under broken fallback is closer to 34 than the 1 originally asserted; both routed as
fixif pushed through classify-work). No fabricated citations, no «would detect» prose, all claims have file:line or command+output backing.Verify-trace
bash .claude/skills/meta-orchestrator/helpers/plan-currency-check.sh→ exit 0, 24 UNTRACKED PRs surfaced. [x] verified via/tmp/stage-5-dogfood-outputs/01-plan-currency.outbash .claude/skills/meta-orchestrator/helpers/priority-score.sh→ exit 0, 33 valid synthetic candidates + 1 phantom*row. [x] verified via/tmp/stage-5-dogfood-outputs/02-priority-score.outbash .claude/skills/meta-orchestrator/helpers/delta-diff.sh <DELTA> "${CANDIDATE_IDS[@]}"→ exit 0, 34NEW-SINCE-LASTlines. [x] verified via/tmp/stage-5-dogfood-outputs/03-delta-diff.outbash .claude/skills/meta-orchestrator/helpers/launch-table-generator.sh "meta-orchestrator-mode-triage-and-planner"→ exit 0,MISSING kickoff:(J5 manifesting). [x] verified via/tmp/stage-5-dogfood-outputs/03a-launch-table.out/tmp/stage-5-dogfood-outputs/04-L*.out<umbrella>-meta-launch/(157 + 64 LOC) per SKILL.md §10. [x] verified viagit status --shortempty +git check-ignore.npm run test:principlesexit 0.Out of scope (explicit)
feedback_no_drive_by_prs. Each gap has a 1-line proposed fix scope in §5 for a follow-up PR..claude/skills/meta-orchestrator/— substrate is read-only target of this dogfood.§1.7 Forward-check applied
This findings doc and PR comply with these existing disciplines (current
.claude/rules/*.mdcontent):phase-research-coverage.md §1.11(file:line:.claude/rules/phase-research-coverage.md:73) — every substrate-behaviour claim in §1 has command + verbatim output + exit code; §2 routing cells cite SKILL.md line numbers; §5 each item has a verbatim repro command. No prose-only claims.ai-laziness-traps.md §3(file:line:.claude/rules/ai-laziness-traps.md:154) — kickoff §5 enumerated T1/T3/T7/T13/T14/T15/T16/T17/T19/T20 + domain-specific T-MMT-5-A/B/C; findings honour them inline (§3 N=1 framing per T-MMT-5-B; §4 T19 cold-review; §5 J1 includes §4 A6 amendment per T16).doc-authority-hierarchy.md §3(file:line:.claude/rules/doc-authority-hierarchy.md:54) — findings doc carries Authoritative-for + NOT-authoritative-for header at top.recommendation-laziness-discipline.md §3(file:line:.claude/rules/recommendation-laziness-discipline.md:22) + parentphase-research-coverage.md §1.12(file:line:.claude/rules/phase-research-coverage.md:86) — every Mode-verdict in §2 cites SKILL.md §5 row + helper-output evidence in the same turn (§2 «Source» column).parallel-subwave-isolation.md §1(file:line:.claude/rules/parallel-subwave-isolation.md:9) — Worker ran in dedicatedgit worktree addat../rules-as-tests-aif-stage-5-dogfoodper kickoff §1.CLAUDE.md PR strategy+feedback_no_drive_by_prs(file:line:CLAUDE.md:87+CLAUDE.md:93) — §5 surfaces bugs as observations; zero substrate fixes applied; no follow-up PRs opened in this Worker session.build-first-reuse-default.md— dogfood ships zero new capability;Prior-art: skipped — dogfood run, no new capability surfacein commit trailer perCLAUDE.mdescape hatch.no-paid-llm-in-ci.md §1— dogfood uses only deterministic bash + subscription-bundled CC session. Zero API-billed calls.§1.7 Backward-check applied
This patch validates substrate produced by these merged PRs (sweep of artefacts under scope):
research/meta-orchestrator-mode-triage-prior-art, merged 2026-05-26T08:21:54Z) — prior-art survey for Stages 2A/2B. §2 routing exercised surfaces this prior-art justified.feat/meta-orchestrator-stage-2b-delta-persistence, merged 2026-05-26T09:35:08Z) — JSON sidecar persistence. §1.4 + §1.9 exercised first-run JSON-absent scenario.feat/meta-orch-stage-2a-discovery-surfaces, merged 2026-05-26T09:34:46Z) — 3 new L1 discovery surfaces. §1.2 emitted 25 openq-§13-* + 3 todo-* entries confirming the surfaces fire.feat/meta-orchestrator-stage-2c-skill-wiring, merged 2026-05-26T10:37:14Z) — SKILL.md §2.5 L3/L4/L5 + routing + ALIAS + principle 19. §1.6 walked L2+L3+L5+ALIAS line-by-line against SKILL.md §2.5.feat/meta-orch-stage-3-delta-read, merged 2026-05-26T13:18:07Z) — delta-diff + reconciliation. §1.4 confirmed first-run semantics (missing JSON → all NEW).feat/meta-orchestrator-stage-4-cli-overrides, merged 2026-05-26T14:54:28Z) — Stage 4 CLI override flags. §1.7 exercisedparse-override-flags.sh "--mode-solo --reason=…"perparse-override-flags.test.ts:97-109contract.meta-orchestrator-mode-triage-and-planner. Post-merge: orchestrator surfaces «umbrella DONE» closure summary.meta-orchestrator-bundle-autonomous) is independent — Stage 5 does NOT block on Bundle Stage 2 verdict..claude/rules/*.md— none modified by this dogfood (verified:git diff --stat origin/staging...HEAD -- .claude/rules/empty).packages/core/principles/*.test.ts— none modified (verified:git diff --statempty).SKILL.md/.claude/skills/meta-orchestrator/helpers/*/templates/*/references/*— none modified (verified:git diff --statempty).