docs(mmp): S4 synthesis + done.md — two-axis parity checklist, root-cause map, prioritised fixes (closes pilot) - #1113
Merged
Merged
Conversation
…ause map, prioritised fixes (closes pilot)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
S4 synthesis closing the
multi-model-pipeline-pilotumbrella: the binding two-axis parity checklist + root-cause map, merging the S2 in-container evidence (merged in #1111), this session's S3 host-side observations, and the §3b banked pipeline-leg rows.One aif task (
e43ce4ea, profileZ.AI GLM-5.2 SDK, off-peak, 552K tokens) carried the whole audit self-referentially — its work product WAS the evidence, and its journey through/arch→/pipeline→/dispatcher→ aif/GLM → shields → harvest → acceptance tested every leg.Also lands
done.md(umbrella-closure convention) and a prioritised fix list (§8) — surfaced, not autonomously opened as PRs.Rollup: 16 ✅ SAME · 4⚠️ WORSE · 0 ❌ BROKEN · 18 ◻️ COVERAGE-LIMITED (each with a named would-reach-it carrier). The BROKEN count moved from 3 → 0 by the §5 Q2 analysis, with an explicit reclassification note: inapplicability-to-this-carrier is not breakage, and the ⚠️ WORSE rows (the real defects) are untouched.
Headline finding —
check-doc-authority.shis registered in the container yet neither checks nor complains:jqis MISSING so it skips by its own code, and its warning goes to bare stderr which the harness never surfaces to the agent. A gate that looks alive in any settings audit while being functionally dead —#warning-nobody-readsin its purest observed form. Fix-pointer: installjqin the image AND re-emit viahookSpecificOutput.additionalContext.Second finding (host-side, new):
harvest-via-api.shmints a fresh commit SHA via the Git Data API, so a cold fidelity audit anchored on the container commit can never satisfy thepr-body-fidelityAudited-SHA == PR headguard. Resolved here by re-anchoring on a byte-identical blob (spec-D10 idempotence); fix-pointer recorded.Parked questions — all three closed: Q1 RESOLVED (root-caused above) · Q3 RESOLVED (plan ran on GLM, not Opus — the D1 whole-pipeline-on-executor intent holds) · Q2 RESOLVED BY ANALYSIS — re-framed from «what was intended» (human-only) to «what should be true for aif to work well» (answerable from what each hook does): 3 of the 5 unregistered hooks are correctly absent for an autonomous container worker —
inject-output-languagewould be actively harmful if registered, since it pushes operator-facing language onto a worker whose repo artifacts must be English — 1 is low-value, and 1 (inject-subagent-context) becomes a real gap only once workers spawn subagents. «Drift» is the wrong frame; the actionable container defect is the missing dependencies (§8 P0).Top actionable (§8 P0): install
jq,gh,python3in the aif container image — one root cause behindcheck-doc-authorityskipping its own check and both pre-commit JSON/YAML validators silently skipping. A registered-but-dependency-less gate is the worst failure mode in this codebase's model: alive in every settings audit, enforcing nothing.Fidelity verdict
FIDELITY: skipped — host-authored S4 synthesis, not an aif-produced stage diff; no
## Provenancesubstrate is declared, so there is no container diff to audit for WHAT-conformance against a worker kickoff scope. The audited artifact of this umbrella was the S2 evidence, which carried its own cold GO verdict in #1111.§1.7 Forward-check applied
Checked the disciplines this synthesis operates under:
no-paid-llm-in-ci.md §1— all observation was session/aif-bound; nothing audit-related enters CI, and the synthesis is plain markdown. file:line evidence:.claude/rules/no-paid-llm-in-ci.md:1ai-laziness-traps.md §2 T14— reachable-but-unfired rows are recorded COVERAGE-LIMITED with a named would-reach-it carrier, never upgraded to SAME. file:line evidence:docs/meta-factory/research-patches/2026-07-23-aif-parity-s4-synthesis.md:1ai-laziness-traps.md §2 T15— §6 records that the auditor reproduced the very failure class under study (ajq-absent probe whose fallback printed a conclusion), caught only by chasing a contradiction with worker evidence. file:line evidence:docs/meta-factory/research-patches/2026-07-23-aif-parity-s2-container-evidence.md:22attention-is-not-a-mechanism.md §1— the headline finding is classified against#warning-nobody-reads; detection in this run came from a named cold agent + a fail-closed gate, not from anyone remembering to look. file:line evidence:.claude/rules/attention-is-not-a-mechanism.md:1doc-authority-hierarchy.md §5— the patch inherits research-patches folder authority and carries a principle-10 scope marker on line 1.§1.7 Backward-check applied
Swept sibling surfaces of class «parity census / audit-evidence artefacts» before landing:
.claude/rules/zcode-parity-doctrine.md— used strictly as a row-set source (T-AIP-D); this synthesis adds the CC-vs-aif axis and does NOT supersede or amend the CC-vs-ZCode census verdicts. file:line evidence:.claude/rules/zcode-parity-doctrine.md:1docs/meta-factory/research-patches/2026-07-23-aif-parity-s2-container-evidence.md(merged docs(mmp): S2 in-container parity evidence — aif/GLM-5.2 SDK carrier run #1111) — extended, not superseded: §2 here merges its row-set with host observations; its own §7 worker-checklist is the direct input. file:line evidence:docs/meta-factory/research-patches/2026-07-23-aif-parity-s2-container-evidence.md:235.claude/orchestrator-prompts/multi-model-pipeline-pilot/kickoff.md§3b banked rows — carried forward verbatim into §2.2 with their original verdicts; the D1 marker row is updated pre-fix→post-fix rather than rewritten. file:line evidence:.claude/orchestrator-prompts/multi-model-pipeline-pilot/kickoff.md:64harvest-via-api×Audited-SHA). file:line evidence:agents/fidelity-auditor.md:74done.mdclosure convention — written in this PR per the umbrella-closure convention. file:line evidence:CLAUDE.md:1