Skip to content

feat(worker): secondmate session runner and IMDS blob lane (R2/R3 PR 2) - #263

Merged
ruby-dlee merged 5 commits into
mainfrom
fm/secondmate-session-runner
Aug 20, 2026
Merged

feat(worker): secondmate session runner and IMDS blob lane (R2/R3 PR 2)#263
ruby-dlee merged 5 commits into
mainfrom
fm/secondmate-session-runner

Conversation

@ruby-dlee

@ruby-dlee ruby-dlee commented Aug 20, 2026

Copy link
Copy Markdown
Owner

What

R2/R3 PR 2 (R2R3-DESIGN.md, section C item 2): the guest-side secondmate session runner and its IMDS blob lane, hermetic-only. No controller, provider, or monitor changes; message-put/message-collect stay in PR 3 and the dispatching monitor in PR 4.

  • bin/fm-secondmate-session.py - stdlib-only runner invoked as the argv of a leg execute under the pinned supervisor's scrubbed environment. One transport class with two backends (imds: IMDS bearer token against the private blob endpoint, x-ms-version 2021-08-06, no client_id since the worker has exactly one user-assigned identity; dir: a fixture directory with identical list/get/put semantics selected by FM_SECONDMATE_BLOB_DIR). The session/ namespace boundary (design D.3) is enforced as a raise INSIDE the transport, above the backend split.
  • Chained outbox: session/out/<seq8>-<sha256>.json, sequence from 1, chain_digest = sha256(prev_chain_hex + content_sha256_hex), genesis previous 64 zeros; the durable tip is re-derived against the store on every start and any gap, reorder, or substitution REFUSES the leg (exit 2, "SECONDMATE SESSION REFUSED: outbox chain is broken..."), never skips or renumbers. The single tolerated divergence is exactly one content-verified entry past the tip (the PUT-then-record crash window).
  • Inbox: content-addressed session/in/<sha256>.json, polled by prefix LIST (default 10s, floor 5), deduped by a durable processed-set on the task disk; replays are no-ops. Attach bundles session/in/attach/<sha256>.bundle are fetched on demand, size-checked against the announcing message before the fetch, then git fetched into the repo.
  • Agent turns: pi --print --session-id <persisted uuid> --session-dir <task disk> per captain message (the D.1-proven resume shape); pi stdout returns as a chained fm.secondmate-message/v1 reply. A message with no honest room before the wall is deferred, not marked processed, and replays next leg.
  • Child intents: bin/fm-secondmate-spawn.pi-ext.ts registers fm_cloud_spawn(kind, brief, model?, effort?) (kind ship|scout) and writes intent FILES to the spool only - no blob or network reach, statically pinned by the suite. The runner sweeps the spool after each turn, validates the closed schema (unknown key or bad kind refuses by name), and emits fm.secondmate-child-request/v1 built from a fixed key set so no home/account/worktree/harness/SKU/repository field can even be expressed, with the parent identity triple and a self_digest over the canonical payload.
  • Per-leg commit bundling: commits over the durable last-bundled tip (initially FM_WORKER_REPOSITORY_GENERATION) upload as session/out/bundle-<seq8>-<sha256>.bundle (cap 256MiB) at leg end and on flush, declared {name, sha256, bytes, commits} in the chained leg summary; zero commits declare none. Tip advances only after upload; pending declarations clear only after the summary that carries them.
  • Leg lifecycle: close control / idle (FM_SECONDMATE_IDLE_SECONDS, 7200) / wall (FM_SECONDMATE_LEG_SECONDS, 14400, exiting with margin before the supervisor's hard timeout) all exit 0 with an fm.secondmate-leg-summary/v1 naming the reason, bundles, and legs_completed.
  • Full CLI/env contract in the module docstring: the supervisor scrubs the environment, so the PR 4 monitor passes configuration as flags; every flag reads an env fallback for direct invocation and tests.

Test evidence

tests/fm-secondmate-session.test.sh (hermetic; registered in test-capabilities.tsv and behavior-test-durations.tsv) drives the REAL runner binary against fixture blob directories and a fixture pi - no runner internals mocked. 16 units, all green (# fm-secondmate-session.test.sh: all assertions passed):

  1. full happy leg with byte-exact canonical reply blob (chain fields included) and close summary
  2. sequence/chain/pi-session-id continuity across two runner invocations
  3. chain tamper (deleted blob N) refuses with the exact string and emits nothing into the tampered store
  4. idle exit emits reason=idle summary
  5. close control emits reason=close with zero agent turns
  6. wall exit emits reason=wall before the hard timeout
  7. wall-adjacent message defers (not processed) and replays exactly once next leg
  8. every pi turn observes PI_CODING_AGENT_DIR bound to the configured agent dir (default /mnt/account/pi-agent and override, dumped by the fixture pi)
  9. leading-dash message text refuses loudly and never reaches the pi argv
  10. child-intent round trip: parent triple, child fields, recomputed self_digest, no binding-shaped key expressible
  11. invalid intents (unknown key, bad kind, cap-overrun brief) refuse naming the exact check, never emitted
  12. namespace boundary: direct unit on the real transport (put/get/list all refuse outside session/) plus the attach code path
  13. processed-set dedupe: a replayed content-addressed inbox message is a durable no-op
  14. commit bundling: real git repo, declared sha256/bytes/commits=1, git bundle verify passes against the dispatched base; zero-commit leg declares no bundle
  15. attach bundle size-checked, fetched, landed; a size lie refuses before fetch
  16. static contract: extension is spool-only I/O; runner pins its refusal strings and REST constants

Mutation proofs (applied transiently, suite rerun, then reverted; clean tree re-verified green after each):

  • M1 skip chain verification (verify_against_store call removed) -> RED: "not ok - tampered outbox must refuse: expected exit 2, got 0" (this doubles as the positive control that the chain-verification line is load-bearing)
  • M2 widen the namespace check (_admit returns early) -> RED: "transport admitted a name outside session/: outcome.bundle"
  • M3 drop the processed-set dedupe -> RED at the chain-continuity unit (5 outbox entries instead of 4: the replayed message re-ran a turn)

Also green after the change: tests/test-suite-seal.test.sh, tests/fm-behavior-shards.test.sh (both consume the edited registries), and bin/fm-lint.sh on the new test file.

Review absorption (2026-08-20 adversarial review)

  • MAJOR: agent_turn now sets PI_CODING_AGENT_DIR from --agent-dir / FM_SECONDMATE_AGENT_DIR (default /mnt/account/pi-agent); the supervisor's scrubbed environment drops the variable and pi would otherwise root agent state at an empty $HOME/.pi/agent. Pinned by unit 8 (fixture pi dumps its environment; default and override both asserted).
  • MINOR: probed pi 0.84.1 locally: it REJECTS a -- end-of-options separator (Error: Unknown options: --, --bogus-flag-xyz), so the separator path is unavailable. Implemented the refusal path instead: captain text beginning with - refuses loudly (message text begins with '-' and cannot ride the pi argv) and never reaches the argv. Pinned by unit 9.
  • Finding 4: --leg-seconds refuses above 21600 (the pinned supervisor MAX_WALL_SECONDS), asserted in the wall unit; the module docstring records the PR 4 monitor contract that supervisor wall_seconds must cover leg_seconds plus the finish-leg budget (git bundle 600s + blob put 600s beyond the capped 300s margin).
  • Finding 3 (accepted edge, deliberate): a leg whose accumulated commits bundle over 256MiB fails the leg-end refusal loudly and will keep failing every subsequent leg end until an operator intervenes on the retained disk (the commits are never lost). Degrading this to a loud leg-summary field instead of a leg failure is deferred; not silently worked around here.

Size

1,649 added lines: ~1,010 are the runner (docstring contract included) and ~540 the 14-unit suite; single concern, no split point that would not orphan the tests from the code they pin.

Honest limits

  • The D.2 IMDS gate PASSED 2026-08-20 from a real worker (private-endpoint DNS 10.42.3.5, slot-identity IMDS token, LIST/PUT/GET green, x-ms-version 2021-08-06; recorded in R2R3-DESIGN.md GATE RESULTS). This suite itself remains hermetic: the dir backend proves the runner's semantics, the live gate proves Azure's side.
  • Real pi behavior (extension loading via -e, session resume under the account/task disk split) is proven by the D.1 probe, not by this suite; the fixture pi proves the runner's contract with any conforming binary.
  • A transport failure mid-leg fails the leg loudly rather than retrying; redispatch is the PR 4 monitor's job.
  • The pi invocation is the exact D.1 shape (--print --session-id --session-dir, plus -e when an extension is staged); flags like --approve are the dispatching monitor's decision in PR 4.

Design reference: R2R3-DESIGN.md sections B.1, B.3, B.4, B.5, C item 2, D (with gate results D.1 passed, D.2 half-resolved).

R2/R3 PR 2 core: bin/fm-secondmate-session.py drives one compartment leg
(IMDS/dir blob transport with the session/ namespace refusal inside the
transport, chained outbox, content-addressed inbox with durable dedupe,
pi turns, child-intent spool sweep, per-leg commit bundling, close/idle/
wall exits). bin/fm-secondmate-spawn.pi-ext.ts is the staged fm_cloud_spawn
tool that writes spool intent files only.
Thirteen units drive the real runner against fixture blob directories and a
fixture pi: byte-exact canonical reply blobs, chain continuity across legs,
chain-tamper refusal, idle/close/wall exits, child-intent round trip and
refusals, the session/ namespace boundary (transport unit + attach path),
processed-set dedupe, commit bundling with git bundle verify, attach
size-check round trip, and the extension/runner static contract.
…equests

A captain message that no longer fits an honest pi turn before the wall is
deferred (never marked processed) and replays on the next leg; a child brief
at the 256KiB bound that cannot fit the framed outbox message refuses that
one intent instead of failing the leg. Both behaviors are pinned by new
hermetic units.
…ap leg seconds

Absorbs the PR #263 adversarial review:
- every pi turn now runs with PI_CODING_AGENT_DIR bound to --agent-dir
  (default /mnt/account/pi-agent, env FM_SECONDMATE_AGENT_DIR); the
  supervisor's scrubbed environment drops the variable and pi would
  otherwise root agent state at an empty HOME/.pi/agent
- captain text beginning with '-' refuses loudly instead of riding the pi
  argv: probed pi 0.84.1 rejects a '--' end-of-options separator
  (Unknown options: --, ...), so the separator path is not available
- --leg-seconds refuses above 21600, the pinned supervisor wall ceiling,
  and the module docstring records the PR 4 monitor contract that
  supervisor wall_seconds covers leg_seconds plus the finish-leg budget
Two new hermetic units (agent-dir default and override observed by the
fixture pi; dash refusal with argv non-reach) plus the ceiling refusal
assertion and static markers.
@ruby-dlee
ruby-dlee merged commit 7f1f76d into main Aug 20, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant