docs(skills): orchestration layering — harness-native orchestration is layer one - #729
Conversation
…tration is layer one Foundation harnesses ship TRAINED orchestration and prose is its native API. The graph layer composes ACROSS harnesses (conserved budget, ledger evidence, resume, heterogeneous models); it must not re-choreograph WITHIN one. Outer coarse, inner maximal. Consequence recorded as the sixth gen4 requirement: the unmeasured-harness case expectation (nodes=3) and v3's held N-roles-N-nodes lesson are both suspect — the layer-aware answer to 'three probes on claude-code' may be ONE node instructed to run three native subagents. gen4 needs a case that encodes this, with the qualifying-harness list owned by the supervisor-lab harness KB.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 9e332e29
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-08-03T21:02:33Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 2 (2 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 60.5s (2 bridge agents) |
| Total | 60.5s |
💰 Value — sound
Adds a substantive doctrinal requirement to gen3's held-ledger gen4 list — correct and in-grain; the live SKILL.md rule it challenges stays live until gen4 ships, which is the repo's intended loop-gated sequencing.
- What it does: Appends one item to the
gen4Requirementsarray insideverifierHoldin skills/agent-graphs/generations/gen3.json:699. It records an orchestration-layering doctrine — foundation harnesses (Claude Code subagents, codex goal-mode, pi extensions) ship trained orchestration and prose is that layer's native API; the runGraph layer composes ACROSS harnesses (conserved budget, ledger evidence, resume, - Goals it achieves: Prevent gen4 from re-deriving the same wrong graph shape by recording, in the ledger that gen4 must read, that 'three probes on claude-code' may correctly be ONE node using native subagents — not three externally-choreographed thin nodes that each pay the harness floor while suppressing the model's trained orchestration. Better system state once merged: the next improvement generation has an expli
- Assessment: Good change on its merits. The doctrine is substantive and correct: re-choreograph fan-out inside a harness that already has trained orchestration wastes the per-node floor and suppresses native behavior, and 'outer coarse / inner maximal' is the right principle. It is recorded in exactly the right place —
verifierHold.gen4Requirementsis the forward-looking requirement list for the HELD gen3 lo - Better / existing approach: none — this is the right approach. Searched skills/agent-graphs/, docs/, and all *.md for the doctrinal vocabulary and for prior gen-requirement lists; the only
gen4Requirementsarray is the one being extended (skills/agent-graphs/generations/gen3.json:693), and the doctrine appears nowhere else. Editing the live SKILL.md 'MUST create N nodes' rule (skills/agent-graphs/SKILL.md:27,86) or the - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error
🎯 Usefulness — sound
A coherent doctrine-record change landing in both the durable prose doc and the per-generation ledger, fitting the established pattern and grounded by a verified case-expectation claim.
- Integration: gen4Requirements is a forward-looking ledger array; IMPROVE.md:54 documents it as the gen4 pointer and bench/src/agent-graphs-gen3.mts writes the file. Nothing parses the array at runtime, which is correct for a requirements record — it is consumed by the next gen4 authoring run, not by code. Reachable via the documented flow.
- Fit with existing patterns: Matches the established ledger pattern exactly: the prior 5 gen4Requirements entries are the same shape (one string per concrete requirement), and landing durable doctrine in IMPROVE.md while recording the per-generation consequence in gen3.json is the intended split. The unmeasured-harness case (cases/unmeasured-harness.json:5) does carry nodes=3, so the doctrine's stated consequence is grounded
- Real-world viability: Pure JSON-string + prose addition with no runtime, concurrency, or error-path surface. The one load-bearing factual claim (nodes=3 expectation) was verified against the case file. Holds up under realistic use because there is no happy path to fail — it is a record.
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🎯 Usefulness Audit
🟡 New ledger entry bundles permanent doctrine with a checkable task into one string [ergonomics] ``
The other 5 gen4Requirements entries (gen3.json:694-698) are each one crisp, independently-closeable requirement. The new entry at gen3.json:699 packs (a) permanent layering doctrine, (b) the outer-coarse/inner-maximal rule, (c) a concrete gen4 case-rewrite task, and (d) harness-KB ownership into one ~600-char string. A gen4 author tracking 'which requirements are done' cannot mark this closed without conflating permanent doctrine (never done) with the case rewrite (checkable). Consider splittin
💰 Value Audit
🟡 Live SKILL.md 'MUST create N nodes' rule stays authoritative until gen4 ships [maintenance] ``
The new requirement makes skills/agent-graphs/SKILL.md:27 ('Identical-Role Parallelism: ... you MUST create N distinct worker nodes') and :86 ('If N parallel instances of a role are requested, create N nodes') suspect for harnesses with trained orchestration, but those lines remain the live, unconditional rule until gen4 promotes. This is by design (loop-gated skill mutation, per commit 5ee60d7) and the requirement is already recorded, so it does not gate this PR — flagging only so a reviewer is
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
| glm | deepseek | deepseek-flash | aggregate | |
|---|---|---|---|---|
| Readiness | 89 | 95 | 89 | 89 |
| Confidence | 65 | 65 | 65 | 65 |
| Correctness | 89 | 95 | 89 | 89 |
| Security | 89 | 95 | 89 | 89 |
| Testing | 89 | 95 | 89 | 89 |
| Architecture | 89 | 95 | 89 | 89 |
Reviewer score is advisory once the run is complete and the verdict has no blockers.
Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.
🟡 LOW Doctrine references a 'supervisor-lab harness-KB row' that is not located in this shot — skills/agent-graphs/IMPROVE.md
Both files assert 'Which harnesses qualify for native fan-out is a supervisor-lab harness-KB row, not a guess.' This is stated as existing infrastructure, but no path or evidence for the harness-KB is given in the changed files. As doctrine this is acceptable (it is a gen4 requirement, not a claim of done work), but if a reader is expected to find this KB row today the reference is dangling. Non-blocking; flagging so gen4 authoring either links the KB or explicitly marks it as to-be-built.
🟡 LOW IMPROVE.md doctrine lacks pointer to the gen4 requirement record — skills/agent-graphs/IMPROVE.md
The header says '(doctrine, gates gen4)' but the section never names where the gen4 consequence lives (gen3.json verifierHold.gen4Requirements, 6th bullet: new one-node+native-fanout case). A reader of IMPROVE.md alone cannot locate the case-design action item. One-line cross-reference would close the loop.
🟡 LOW Casing inconsistency for TRAINED between the two files — skills/agent-graphs/generations/gen3.json
The gen3.json bullet writes 'ship TRAINED orchestration' (all-caps TRAINED) while IMPROVE.md line 41 writes the same phrase as 'ship trained orchestration' (lowercase). Same doctrine, two casings. Cosmetic only — pick one for consistency. Fix: lowercase 'TRAINED' in the gen3 bullet to match IMPROVE.md.
🟡 LOW New doctrine-motivated case could collide with blind-authoring holdout rule — skills/agent-graphs/generations/gen3.json
This requirement is explicitly motivated by measured failure (unmeasured-harness holdout scored 0.5 on 3/5 v3 reps; the held N-roles-N-nodes rule), while gen4Requirements also mandates 'holdout refresh: cases authored blind ... never failure history'. If gen4 encodes the doctrine by mutating the existing holdout case unmeasured-harness, it reproduces the exact case-design contamination pattern the verifier invalidated gen3 for. The bullet says gen4 'needs a case' but not whether it is a fresh blind-authored case vs a rewrite of the suspect holdout. Recommend stating the new case is authored blind to measured failure and added as a new id, not a mutation of unmeasured-harness.
tangletools · 2026-08-03T21:06:06Z · trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 4 non-blocking findings — 9e332e29
Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-08-03T21:06:06Z · immutable trace
Doctrine correction from review: foundation harnesses ship trained orchestration (Claude Code subagents, codex goal-mode, pi extensions) and prose is that layer's native API — instructing a claude-code node to fan out subagents invokes an in-distribution capability, not vibes. The graph/script layer exists only for what no single harness provides: cross-harness composition, one conserved budget across the tree, ledger evidence, resume, heterogeneous models.
Rule: outer layer coarse, inner layer maximal — one harness-sized node told to use its native fan-out beats N externally-choreographed thin nodes that each pay the harness floor while suppressing the model's trained orchestration.
Recorded as the sixth gen4 requirement, because it makes two current artifacts suspect: the
unmeasured-harnesscase expectation (nodes=3) and v3's held N-roles-N-nodes lesson — the layer-aware answer to 'three probes on claude-code' may be one node running three native subagents. Which harnesses qualify is a supervisor-lab harness-KB row, not a guess.