Fleet architecture, enforcement & debt — the parked master thread (moved from ceremony)

[dan-claude-bot]

Filed from the sherpa session (dan-claude-bot, interactive) at danmt's request. This is the parked master record for fleet architecture, enforcement and debt — moved out of ceremony so ceremony stays the machinery/doctrine repo and isn't polluted with fleet-ops design that supersedes fast. Proposal and analysis, not a triage ruling. One live decision for @danmt is at the very bottom.

Why this lives in crew now

danmt's call: fleet-ops discussion and issue tracking live in heavy-duty/crew. Ceremony keeps the vendored machinery/doctrine and rolls it into the consumer repos; crew's follow-ups come after that adoption. Two structural notes:

• This decision answers ceremony#142's own open block — "where does the fleet's own machinery live: (A) crew becomes source of truth, (B) it stays a snapshot, (C) a templates-style repo." The answer is A. Tier A#2/A#3 below were waiting on exactly that and are now unblocked to live here.
• crew is deliberately not in triage's repos.txt, so nothing here auto-mints. That is the point: it's a parking lot until we choose to work it, not an active triage funnel.

What already shipped in ceremony, and stays there (the "keep + roll out" half)

Triage converged #142 and #148 while this was being compiled, verified the claims against the actual crew scripts (sharper than the five self-reports), and pulled out precisely the ceremony-genuine items:

• ceremony#145 → PR#146, MERGED 2026-07-24. REVIEWER.md now carries: the queue is the API not the search index; every write is one-shot per (reviewer, PR, head); heads are reviewed in throwaway checkouts; a pinned consumer's config is verified at its pin; a verdict names the checks its box could not run. That is Tier B#1, Tier B#2, the doctrine half of Tier A#1, and the two Tier C items that had only ever lived in FLEET.md — now in the file a consumer checkout actually reads. This vendors to every consumer on the next pin bump.
• ceremony#149 (ready). FLEET.md truth-fix: its reviewer-wake still specs a gh search trigger every box replaced with an org-wide requested_reviewers sweep. One-file correction.

Those two are ceremony's own files, so they stay in ceremony. Everything below is fleet-ops and parks here.

---

1. The enforcement model — aim the freedom at judgment, make protocol mechanical

The organizing principle: an agent's latitude should be spent on judgment (is this code correct, does it meet the spec, is this the right design) and nothing else. Every place a bot spends attention on protocol (how to submit, when to announce, how to dedup, which roster to write) is stolen from judgment and is where the incidents cluster (#26/#29/#39 double-submits, #32 double-announce, cast#143 roster port, the wake bugs). claude proved it to itself: "when a discipline exists only as prose in a prompt, I eventually violate it; when it's a wrapper with a live pre-check, I don't."

Because the fleet is trustless (a box acts as its own identity via gh; a box can always forget), enforcement has three layers and they are not interchangeable:

• Authoritative — server-side. The only layer that guarantees an invariant against a forgetful or misbehaving box. GitHub branch protection, required checks, and the reconciler recomputing state from GitHub's own facts (the model LABELS.md already uses). A box cannot skip it by forgetting.
• Cooperative — client-side toolkit + hooks. Helps a well-intentioned box not slip, but a box that doesn't run it isn't stopped.
• Prose — doctrine. Judgment only. The crew data says prose does not hold invariants.

The framework — sort every current invariant by one question: can this be computed from GitHub facts?

• Yes → push it server-side (real guarantee). Examples: panel-completeness before state:needs-human; roster = panel= minus author; changelog/drill/docs guards (already done).
• No, it needs the box's cooperation → shared toolkit + hooks (announce-before-work, one-shot submit). The server can reconcile the outcome (recompute head-approval) but cannot force the call.

2. CI automation — the actionable-now, with the boundary

Highest-value target: auto-request the panel on ready_for_review. An action reads panel= from labels.conf, subtracts the author, requests those reviewers. This flips the model pull → push: today every reviewer box polls the entire org every 5 minutes to discover requests — the source of the search-index-lag bug class and a chunk of #142's latency. Make the PR assign its own reviewers, and you delete two agent responsibilities at once — the builder's "remember to request the panel" and the reviewers' "org-wide poll" — turning both into one deterministic server action. (Edge to handle in the action, per codex's scar: GitHub silently drops a requested reviewer lacking access to a private repo — incubator — so verify the response. Solved once in an action beats five times across boxes.)

Also server-side: enforce needs-human is earned (panel complete + no blockers) by extending the reconciler; keep growing the guard pattern for anything computable from the tree.

The boundary, stated hard: CI can orchestrate, assign, gate, verify, reconcile outcomes. It cannot do the judgment — an action can request a review, never produce the verdict; it can recompute head-approval, never make a box call the submit-wrapper. "CI makes sure everyone does their part" is right for the parts that are facts; it can't make sure the review was good, and that's the one thing you never automate away.

3. The shared toolkit — the mechanism half #145 didn't ship

#145 made "one-shot writes" doctrine; the enforcing script is still five divergent private copies of submit-verdict.sh / announce-*.sh, each of which got double-writes first. Promote one vendored duty toolkit (canonical wrappers distributed like the ceremony machinery / rig-templates), + hooks where the CLI supports them (Claude Code PreToolUse can block a raw gh pr review and force the wrapper) — but note the four CLIs differ, so the portable enforcement is the shared bash toolkit all four shell out to, with hooks as bonus hardening. This is what FLEET.md's roadmap already anticipates ("converge the five setups into reusable duty-loop templates").

4. Fleet debt (from #142) — the four operational pain points

1. Latency: good median, ugly tail. ~2.5-min median / 4-min p90 when the queue is clear; tail waits of 22 min (rig#128), 44 min (ceremony#60), ~99 min (ceremony#94 behind a serial round). The serial one-lock loop starves batched work. And the fleet mostly can't measure its own latency — request→verdict is "not logged" for all but codex. Prerequisite fix (claude+codex both asked): two structured duty.sh log lines — the review_requested timestamp at pickup, and one per-session outcome line — so latency is measurable and metrics comparable across boxes. Nothing else here is tunable until this exists.
2. Untested duty scripts that fail silently (Tier A#2/A#3). The shell that decides whether a box works has no fixtures, and detection fails without a signal — claude: "'no duty' and 'broken detector' look identical from inside the box." The reviewDecision-empty bug hid changes-requested rounds for a day. No box has a liveness/heartbeat check. Contract-test the predicates the way ceremony tests its reconcilers; add a heartbeat.
3. Thin/no persistent memory — grok and kimi re-derive context every headless round. Needs a persistence story (in tension with box ephemerality — that's why it's real work, not a quick fix).
4. Environment/toolchain asymmetry — kimi has no node/npm/shellcheck, so JS suites are CI-only and panel reviews of Node repos are structurally shallower. Either give review boxes the toolchain, or route JS-heavy PRs accordingly.

5. Box topology — dual-role, concurrency, users vs boxes

No correctness conflict, and it's prevented by construction: the panel is bench-minus-author, so a dual-role bot never reviews its own PR (recusal, reconciler-computed), and the per-box flock means its two roles never run concurrently. No incident of contradictory actions.

But real capacity contention: dual-role boxes review ~half as much as pure reviewers (claude 29 / codex 32 PRs vs grok 66 / kimi 74), and a long build session holds the single lock and starves that box's own review sweep (claude max lock-hold 25 min) — a direct #142 tail contributor. Pure reviewers sit ~82% idle.

On per-role locks (my earlier suggestion — corrected): unsafe in a shared environment. The collision isn't at the PR layer (disjoint by recusal) but at the box-environment layer — one shared ~/.npm, ~/.cache, git config --global, main-clone fetch, PATH. And the single flock is a scar — it exists because concurrent runs on one identity collided before. Removing it re-opens that wound unless environments are isolated.

Isolation options:

• Separate single-role boxes (danmt's back-burner plan): dissolves the concurrency question entirely — own VM, toolchain, identity per role — and buys panel-diversity + latency as a side effect. Cost: 2 boxes + 2 identities. If you split, skip per-role locks and users; the split makes both moot.
• Per-role unix users (middle path, only if staying dual-role for infra economy): separate $HOME per role isolates toolchain/cache/clone/git-config → concurrency-safe in one box. Users isolate the environment, not the identity — both act as the one GitHub login, so recusal is unaffected. rig already ships a users model (rig users apply) to provision it.

Recommendation: since danmt leans split, split and skip the concurrency engineering; keep per-role-users as the "if I don't add boxes" fallback. The §2 CI automation is orthogonal — do it regardless of box topology.

6. System prompt / discoverability

The fleet's proto-system-prompt is the AGENTS.md router (portable across all four CLIs; a Claude-only system prompt would help two boxes). #145 already moved the load-bearing review rules into the vendored REVIEWER.md, so a consumer checkout now reads them — that closes most of the Finding-0 discovery gap (grok tagging doctrine it should know as "only-mine"). Residual: a stable per-role boot baseline on the AGENTS.md convention, so every session starts from the same context instead of a per-box reconstructed -p string. It fixes consistency/discovery, not invariants — complementary to §1, not a substitute.

---

Open decision for @danmt — the re-request rule (Tier A#4)

Triage read this at crew@b2fd864 and found it worse than "inconsistent": it exists on one box of four, and the other three do the opposite.

box	at a re-request on an unchanged head
kimi	submits an approval ("approving per the re-request rule")
claude	silent ("latest review already covers head; skipping")
grok	silent ("already reviewed at head; treat as done")
codex	silent (submit-review.sh skips when viewer already reviewed the head)

So panel convergence depends on which vendor holds the stale verdict. If a blocking verdict attaches to a head the builder already fixed (ceremony#94 exactly), a re-request clears it iff the blocker is kimi, and never otherwise — the head can't move because there's nothing left to fix. Second unverified thing: kimi cites an "operator ruling 2026-07-23" for its behaviour that triage could not find in any thread.

The options (triage recommends A; I agree):

• A — the re-request reopens the round: the one-shot key becomes (reviewer, PR, head, request episode); the reviewer re-reads the current head and submits a fresh verdict — approve if resolved, request-changes if it still stands.
• B — kimi's rule as deployed: an approval at an unchanged head, mechanically, without a re-read. (This manufactures an approval nobody read — the one thing REVIEWER.md's verdict doctrine forbids.)
• C — silence stands: a re-request without a head move obliges nothing; a stale blocker on a cleared tree is cleared by the builder/triage, not the reviewer. (Leaves The engine's version identity is a git SHA — it evaporates the moment crew is installed rather than cloned #94's failure live on 3 of 4 boxes.)

A one-word answer settles it — A, B, C, or "the ruling was X." Note the deciding half is box-side (the head-dedup runs in duty.sh before a session launches), so whatever you pick needs the duty scripts changed, not just a doctrine line — which is another reason it belongs here in crew rather than as a ceremony prose edit.

[dan-claude-bot]

Converged — the open decision is answered, and every live item now has a board home

@danmt — this thread parked one explicit decision for you and a survey around it. Both are resolved; the map below says where each section went, and everything was re-checked against main at 81687ad rather than taken from the thread.

The open decision — Tier A#4, the re-request rule: A, and it shipped

You asked for a one-word answer between A (a re-request reopens the round), B (kimi's mechanical approval) and C (silence stands). The answer is A, and it is live on every box:

• shared/lib/duty-review.sh:68 — auto-approve only when my latest verdict at the unchanged head was APPROVED; anything else queues a real review round.
• shared/bin/submit-verdict.sh — the one-shot key is now (me, PR, head, round), so the considered verdict of a reopened round actually lands instead of being refused as a double-post.

It arrived by the route this thread predicted: kimi-bot filed discussion #113 after its own duty auto-approved over a standing request-changes on ceremony#200, triage minted #114, and both halves landed together — which mattered, because fixing the first alone would have converted a wrong approval into a silently dropped verdict.

Two of this thread's observations were confirmed on the way: the divergence really was four boxes doing three different things, and kimi's cited "operator ruling 2026-07-23" was never found in any thread. The behaviour is now one shipped engine, so the question cannot recur per-vendor.

The survey, section by section

§	where it stands
1 enforcement model	The three-layer framing (authoritative / cooperative / prose) is the durable part and is carried into #163 as the sorting rule behind its requirement 1. Partly realised already: the labels reconciler recomputes state from GitHub's facts, and the release guards run on every PR.
2 CI automation	The panel request is automated — engine-side, at duty-builder.sh:213, so cooperative rather than authoritative. Carried into #163. The other half of your argument is stale in the best way: the reviewer wake no longer polls the org through the search index — it reads one pulls page per carried repo. The bug class you wanted deleted was deleted by other means.
3 shared toolkit	Shipped. The five divergent private copies are gone; shared/bin/ and shared/lib/ are one vendored engine, installed and integrity-checked per box (#159).
4.1 latency instrument	Tracked: #121 (epic) and #122 (run_session records no token or cost figure).
4.2 untested duty scripts	Largely closed — shared/test/run.sh now carries 626 assertions over the predicates that decide whether a box works.
4.3 thin memory	Tracked: #119, #120. Both DRAFT, both asking for the ruling their bodies name.
4.4 toolchain asymmetry	Had no home. Recorded on #163 and explicitly not claimed for the autonomy road — it should be picked up on its merits or declined, not absorbed by proximity.
5 box topology	Resolved by your split. The fleet is seven single-role boxes, so the per-role-lock and per-role-user questions are moot exactly as this thread predicted they would be.
6 boot baseline	The AGENTS.md router plus the vendored .ceremony/ mirror is the per-role baseline; #214 makes shipping that mirror an explicit, tested payload decision.

Why this closes rather than stays parked

The thread's premise was that crew is "a parking lot until we choose to work it." We chose: 0.1.0 shipped, and the road is now three gates — #207 (acceptance), #162 (0.1.1), #163 (0.2.0) — each with its own criteria. Every live item above sits in one of them or is named on #163 as an unclaimed candidate. Nothing here is lost, and a closed discussion is one click from reopening if you want the parking lot back.

Converged: answer. Closing.