Skip to content

docs(mmp): S4 synthesis + done.md — two-axis parity checklist, root-cause map, prioritised fixes (closes pilot) - #1113

Merged
artyhoo merged 1 commit into
stagingfrom
feature/mmp-s4-closure
Jul 23, 2026
Merged

docs(mmp): S4 synthesis + done.md — two-axis parity checklist, root-cause map, prioritised fixes (closes pilot)#1113
artyhoo merged 1 commit into
stagingfrom
feature/mmp-s4-closure

Conversation

@artyhoo

@artyhoo artyhoo commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Summary

S4 synthesis closing the multi-model-pipeline-pilot umbrella: the binding two-axis parity checklist + root-cause map, merging the S2 in-container evidence (merged in #1111), this session's S3 host-side observations, and the §3b banked pipeline-leg rows.

One aif task (e43ce4ea, profile Z.AI GLM-5.2 SDK, off-peak, 552K tokens) carried the whole audit self-referentially — its work product WAS the evidence, and its journey through /arch/pipeline/dispatcher → aif/GLM → shields → harvest → acceptance tested every leg.

Also lands done.md (umbrella-closure convention) and a prioritised fix list (§8) — surfaced, not autonomously opened as PRs.

Rollup: 16 ✅ SAME · 4 ⚠️ WORSE · 0 ❌ BROKEN · 18 ◻️ COVERAGE-LIMITED (each with a named would-reach-it carrier). The BROKEN count moved from 3 → 0 by the §5 Q2 analysis, with an explicit reclassification note: inapplicability-to-this-carrier is not breakage, and the ⚠️ WORSE rows (the real defects) are untouched.

Headline finding — check-doc-authority.sh is registered in the container yet neither checks nor complains: jq is MISSING so it skips by its own code, and its warning goes to bare stderr which the harness never surfaces to the agent. A gate that looks alive in any settings audit while being functionally dead — #warning-nobody-reads in its purest observed form. Fix-pointer: install jq in the image AND re-emit via hookSpecificOutput.additionalContext.

Second finding (host-side, new): harvest-via-api.sh mints a fresh commit SHA via the Git Data API, so a cold fidelity audit anchored on the container commit can never satisfy the pr-body-fidelity Audited-SHA == PR head guard. Resolved here by re-anchoring on a byte-identical blob (spec-D10 idempotence); fix-pointer recorded.

Parked questions — all three closed: Q1 RESOLVED (root-caused above) · Q3 RESOLVED (plan ran on GLM, not Opus — the D1 whole-pipeline-on-executor intent holds) · Q2 RESOLVED BY ANALYSIS — re-framed from «what was intended» (human-only) to «what should be true for aif to work well» (answerable from what each hook does): 3 of the 5 unregistered hooks are correctly absent for an autonomous container worker — inject-output-language would be actively harmful if registered, since it pushes operator-facing language onto a worker whose repo artifacts must be English — 1 is low-value, and 1 (inject-subagent-context) becomes a real gap only once workers spawn subagents. «Drift» is the wrong frame; the actionable container defect is the missing dependencies (§8 P0).

Top actionable (§8 P0): install jq, gh, python3 in the aif container image — one root cause behind check-doc-authority skipping its own check and both pre-commit JSON/YAML validators silently skipping. A registered-but-dependency-less gate is the worst failure mode in this codebase's model: alive in every settings audit, enforcing nothing.

Fidelity verdict

FIDELITY: skipped — host-authored S4 synthesis, not an aif-produced stage diff; no ## Provenance substrate is declared, so there is no container diff to audit for WHAT-conformance against a worker kickoff scope. The audited artifact of this umbrella was the S2 evidence, which carried its own cold GO verdict in #1111.

§1.7 Forward-check applied

Checked the disciplines this synthesis operates under:

  • no-paid-llm-in-ci.md §1 — all observation was session/aif-bound; nothing audit-related enters CI, and the synthesis is plain markdown. file:line evidence: .claude/rules/no-paid-llm-in-ci.md:1
  • ai-laziness-traps.md §2 T14 — reachable-but-unfired rows are recorded COVERAGE-LIMITED with a named would-reach-it carrier, never upgraded to SAME. file:line evidence: docs/meta-factory/research-patches/2026-07-23-aif-parity-s4-synthesis.md:1
  • ai-laziness-traps.md §2 T15 — §6 records that the auditor reproduced the very failure class under study (a jq-absent probe whose fallback printed a conclusion), caught only by chasing a contradiction with worker evidence. file:line evidence: docs/meta-factory/research-patches/2026-07-23-aif-parity-s2-container-evidence.md:22
  • attention-is-not-a-mechanism.md §1 — the headline finding is classified against #warning-nobody-reads; detection in this run came from a named cold agent + a fail-closed gate, not from anyone remembering to look. file:line evidence: .claude/rules/attention-is-not-a-mechanism.md:1
  • doc-authority-hierarchy.md §5 — the patch inherits research-patches folder authority and carries a principle-10 scope marker on line 1.

§1.7 Backward-check applied

Swept sibling surfaces of class «parity census / audit-evidence artefacts» before landing:

@artyhoo
artyhoo enabled auto-merge (squash) July 23, 2026 20:50
@artyhoo
artyhoo merged commit 09863c1 into staging Jul 23, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant