fix(offload-test): a stage probes for its checkout before it runs

[debuggingfuture]

Problem & Insight

A staged run's retry policy is still a lie after #127, for a reason #127's isolation cannot reach on the current runtime.

The runtime already names the failure precisely:

working directory '<dir>' was missing at exec time —
the checkout did not survive to this step (container recycled)

Container disk is ephemeral. A staged run spanning forty minutes of durable steps is long enough for the instance behind it to be recycled, and the next stage then execs into a directory that is gone. That is raised as ExecFailed — exactly the class retryOn retries. So the platform re-ran the same command in the same missing directory three times and reported a failure about a missing directory rather than anything about the code. The retry could never have worked: the thing it needed was the thing that was gone.

#127's isolation does not fix this, because the CF runtime cannot give a run two containers. acquire: () => Effect.succeed({ id: sandboxId }) returns one id per execution, and exec routes through a box client closed over at Layer build — the container handle a caller passes is not what selects the container. Turned on for five stages, all five failed in under five seconds with CheckoutFailed: they did not get five containers, they got one, and raced to wipe each other's checkout (git clone clears its target directory first). Per-stage containers need the runtime to route by handle — a real change to the "V0 = one container per execution" design, and a separate one.

Take

A stage stops assuming its workspace. It probes test -d <dir>/.git inside its own retryable step and re-clones when the probe fails.

• Happy path: one extra exec, about a second.
• Recycled container: a clone and an install — which is what the stage was going to need anyway, and the difference between a green stage and three identical failures.
• Works on today's runtime, because it re-clones into the same container rather than asking for a second one.

Isolated mode keeps acquiring per stage (unchanged from #127), so the same code path serves both once the runtime can honour it.

The single-exec path is deliberately untouched. Its contract is pinned byte for byte by the suite, and its exposure is one step rather than five — the same fix belongs there, but not in a change whose effect on other consumers cannot be observed from here.

Key actions

• A test that pins the recovery: probe answers non-zero → two clones, the command still runs, the run goes green
• Four existing assertions updated for the probe, including the positional timeout checks
• Full suite green — 2240 tests, typecheck, oxlint
• Routing exec by the container handle — the prerequisite for stageConcurrency > 1 actually working — stays open

[flaredispatch-fractalboxdev]

AI code review — 💬 Comment

Risk tier: lite · 0 critical · 2 warnings · 0 suggestions

Reviewers: security 0 · code-quality 0 · performance 1 · documentation 1

1. ⚠️ Warning — Rebuilt shared workspace is not retained for later stages

📍 runs/offload-test.ts:777-790

When 'ensureWorkspace(shared)' detects a recycled container, it acquires a replacement workspace only in the local 'ws' variable. The immutable 'shared' reference remains pointed at the missing container, so every subsequent shared stage probes the stale container and performs another clone/install. Retain the rebuilt workspace for later stages (with appropriate synchronization for concurrent stages) to avoid repeated expensive workspace setup.

2. ⚠️ Warning — README overstates checkout probing for isolated stages

📍 runs/README.md:278-280

The documentation says every stage runs 'test -d dir/.git' and re-clones when that probe fails, but isolated stages do not run this probe; they acquire a fresh workspace directly inside the retryable step. Clarify that probing applies to shared-stage workspaces, while isolated stages rebuild their workspace on each retry.

📋 View full logs & reviewed diff ↗