Define the local Workspace host topology

[taras]

Story

As an Executable.md maintainer, I want one evidence-backed local workflow topology, so implementation of #218 does not depend on unresolved choices between Deno, workerd, FUSE and Cloudflare-hosted Durable Objects.

Decision

The first local xmd workflow host uses this topology:

xmd CLI and lifecycle (Deno)
└── WorkflowRun SQLite
    ├── filtered journal
    ├── versioned Workspace filesystem through Cloudflare DOFS
    ├── Repository and Worktree metadata
    └── Agent-session mappings

• Deno owns SQLite directly. The local host reuses Cloudflare's DOFS filesystem layer behind a provider-neutral Workspace capability.
• The journal and Workspace share the run database so one Workspace-local mutation and its journal result can commit in one SQLite transaction. They remain logically separate retention domains.
• Native Agent processes inspect disposable read-only materializations of a logical Workspace root. There is no writable FUSE or write-back synchronization path.
• The default local binary does not bundle workerd. A Cloudflare-hosted or later workerd provider may implement the same contextual contracts without changing documents.
• Worker JavaScript is deferred from the first local capability set.
• Worker Shell is included. It is the Cloudflare Workspace Shell capability implemented by just-bash, Cloudflare's Workspace filesystem adapter and a Deno Worker boundary—not native Bash or arbitrary native commands.

The CLI reserves a host-neutral lifecycle surface but exposes no remote-host selector initially.

Why this topology

The three spikes removed the topology uncertainty:

• Spike: bundle workerd into the compiled XMD host #347 / PR 🔬 Spike: bundle workerd into the compiled XMD host (#347) #348 shows the bundled-workerd path and its cost. It remains useful provider evidence but is unnecessary for the initial local host.
• Spike: host Cloudflare DOFS directly in Deno with SQLite and FUSE #349 / PR 🔬 Spike: host Cloudflare DOFS directly in Deno with SQLite (#349) #350 shows that Deno can own SQLite and reuse DOFS directly, preserving filesystem state without launching workerd.
• Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno #351 / PR 🔬 Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno (#351) #353 shows the available Worker Shell and Worker JavaScript boundaries. Shell containment holds; JavaScript termination and compiled-artifact constraints make it premature.
• Spike: make Worker Shell effects transactional with the workflow journal #357 / PR 🔬 Spike: make Worker Shell effects transactional (#357) #362 proves one Worker Shell invocation can share its SQLite effect transaction with the journal, roll every known failure path back, survive a host crash without partial publication and fence stale Worker requests.

Production #363 preserves that proof: one authoritative host-owned DOFS connection per workflow database, serialized local effect transactions and forceful Worker termination for CPU-bound programs.

Ownership boundaries

• WorkflowRun and expansion identity are established before Workspace attachment.
• The Deno workflow adapter owns SQLite, DOFS attachment and local effect transactions.
• One authoritative host-owned DOFS connection serves each workflow database and serializes its Workspace-local effects.
• Shared production modules use contextual filesystem, cwd, process, Git, forge and Agent APIs; they contain no Cloudflare types or runtime detection.
• A completed replay returns from retained history without attaching DOFS, Agents or external providers.
• A partial replay attaches the same run database and continues from its retained Workspace frontier.
• Missing or corrupt authoritative Workspace state is an explicit unrecoverable failure; the host never substitutes an empty Workspace while claiming continuation.

Acceptance

• architecture.md records this topology and its invariants.
• specs/workflow-workspace-spec.md records the observable workflow contract that consumes it.
• Implement retained Workflow Workspace restoration #218 uses the implicit, run-owned Workspace and one local SQLite retention boundary.
• The local provider can atomically commit a Workspace mutation, resulting logical root and filtered journal result.
• The topology needs neither writable FUSE, native subprocess access nor bundled workerd.
• Worker Shell preserves Spike: make Worker Shell effects transactional with the workflow journal #357's transaction fencing, rollback, crash recovery and forceful cancellation boundaries.
• The provider seam remains capable of hosting the same contract on Cloudflare later.

This story closes when the architecture/specification PR merges. It selects the topology; #218 and its child implementation stories build it.

Dependencies

• Identity: Add contextual workflow runs with pinned commits #289 / merged PR 💥 Add expansion identity and pinned-commit workflow runs #341
• Bundled workerd evidence: Spike: bundle workerd into the compiled XMD host #347 / PR 🔬 Spike: bundle workerd into the compiled XMD host (#347) #348
• Deno DOFS evidence: Spike: host Cloudflare DOFS directly in Deno with SQLite and FUSE #349 / PR 🔬 Spike: host Cloudflare DOFS directly in Deno with SQLite (#349) #350
• Worker boundary evidence: Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno #351 / PR 🔬 Spike: run Cloudflare Worker Shell and Worker JavaScript natively in Deno (#351) #353
• Worker Shell transaction proof: Spike: make Worker Shell effects transactional with the workflow journal #357 / PR 🔬 Spike: make Worker Shell effects transactional (#357) #362
• Production Worker Shell integration: Add transactional Worker Shell to workflow Workspaces #363
• Workspace implementation: Implement retained Workflow Workspace restoration #218
• End goal: Document adversarial implementation workflow WIP #181

[taras]

Candidate D: bundled local Workspace host

The design session identified a fourth topology that needs evidence before this issue can be decided:

• the compiled xmd artifact contains the Worker bundle, workerd configuration, and a platform-specific workerd executable;
• XMD materializes and supervises workerd as a loopback sidecar;
• Durable Object SQLite state lives in an explicit persistent XMD data directory; and
• XMD reaches the Worker through an internal transport while preserving a single-download user experience.

This is a single-file distribution but a multi-process runtime. It is not a Durable Object running inside Deno, and standalone workerd must not be assumed to provide Cloudflare's distributed service or every @cloudflare/computer execution backend.

#347 now proves or rejects this topology. Treat the topology decision here as blocked on that evidence. If the filesystem works but one or more execution backends do not, this issue must decide whether a filesystem-only local provider is a coherent first slice or whether Workspace remains Cloudflare-hosted.

[taras]

Evidence from spike #347 (PR #348) relevant to the topology decision — the selection remains yours; this is what the measurements support:

The bundled-local-host option is technically feasible, end to end. A single compiled executable carries workerd + a Worker + a Durable Object, runs with no external runtime installed, and preserves both DO SQLite state and the @cloudflare/computer 0.1.1 filesystem Workspace across full process restarts against an explicit state directory, with identity and state-dir isolation. All three workspace.runtime backends work under standalone workerd — worker-shell and worker-javascript unconditionally; Cloudflare Container only with a local Docker daemon (x64 computerd under Rosetta on Apple Silicon, and workerd leaks containers on SIGTERM).

Costs and caveats a selection (or a limited-subset selection) would carry:

• Size: +109 MB uncompressed (~30-40 MB compressed) per release artifact for the platform's workerd. No musl/Alpine build; Linux glibc 2.35+.
• Stability surface: workerd's localDisk DO persistence is schema-marked experimental (it is also what miniflare/wrangler ship on), and Computer 0.1.1 is an explicit preview whose storage schema refuses downgrades. Both are pinned exactly in the spike; an upgrade story is required for anything durable.
• Supervision gaps we'd own: SIGKILL orphans workerd; no state-dir locking; container reaping is the host's job.
• Not representative of deployed Cloudflare: no hardened sandbox (workerd's own README warning — local execution runs at user privilege one V8 bug away), KV/R2/Queues/Cache/D1 bindings are unimplemented protocol shells, single-process DO ownership.

What the evidence does NOT cover: transports/auth to a deployed Workspace (option C concerns), embedder-owned host ergonomics (option A), or any <Workspace> contract work — per #347's non-goals.

If the decision lands on the bundled-local-host (or a subset like filesystem + worker backends only, which needs no Docker), the spike's supervision, packaging, and pinning mechanics in spikes/347-workerd/ are directly liftable.

[taras]

Candidate E: Deno-local SQLite Workspace

#349 now evaluates a fifth topology:

• Deno owns a file-backed SQLite Workspace directly;
• Cloudflare's MIT-licensed DOFS schema and filesystem operations are reused through their structural SQL-storage seam;
• XMD filesystem components use the Workspace API in-process; and
• native commands reach the same state through Cloudflare's FUSE adapter or, if evidence supports it, the explicitly weaker userspace shim.

This could remove workerd and the second V8 runtime from the local host. It also gives up the Workers execution environment and must solve native FUSE packaging, privileges, platform coverage, and subprocess security.

The source audit found that packages/dofs already adapts to node:sqlite in Cloudflare's Node tests, but the adapter is memory-only and both @cloudflare/dofs and @cloudflare/computerd are private/unpublished packages. The repository is MIT licensed, so reuse is legally possible; the spike must prove a maintainable dependency or vendoring boundary. The published @cloudflare/computer@0.1.1 entrypoint cannot currently be imported directly by Deno because its graph contains a cloudflare: module.

Treat the topology selection as blocked on comparing #349 with #347 / PR #348.

[taras]

Second evidence set: spike #349 (PR #350) — the Deno-local DOFS/SQLite topology — is complete, giving this decision both sides. The comparison on the ten axes lives in spikes/349-dofs/evidence/COMPARISON.md; condensed:

Deno-local DOFS (#349): 110 MB artifact, 0.08 s per op, one process, no materialization cache, identity = database file path, Cloudflare's exact filesystem schema via a pinned vendored copy (3-line upstream diff drafted to eliminate the vendoring). Costs: none of Computer's execution backends; no isolation whatsoever; native-subprocess access is a dev-only shim today (real FUSE needs a Node sidecar until a Deno uv-polyfill gap closes — repro committed — and is Linux-only in practice; darwin-arm64 is a dead end at the pinned fuse-native).

Bundled workerd (#347/PR #348): the real published package in the real runtime — all three execution backends work; behaviorally closest to hosted Cloudflare. Costs: 191 MB, supervised child with recorded teardown gaps, Docker for anything native.

The spikes' joint recommendation is limit: select Deno-local DOFS as the substrate for the filesystem-only <Workspace> (the thing #218 needs first — same schema, ~3× cheaper per op, no supervision surface), keep bundled workerd as the documented option for the Computer-execution-backend subset, and treat native-subprocess workspace access as explicitly limited in the near term. Both paths share the schema and sync protocol, so neither forecloses hosted Cloudflare or each other.

The selection is yours; both PRs (#348, #350) carry the reproducible proofs either way.

[taras]

Third and final evidence set: spike #351 (PR #353) measures what imperative execution the Deno-local topology can carry on its own. With #347 (bundled workerd) and #349 (Deno-local DOFS), the decision now has both topologies and the execution question answered. Full comparison in spikes/351-worker-backends/evidence/COMPARISON.md.

The topology decision separates into two questions.

1. Is Deno-local DOFS sufficient for the declarative durable core? — Yes, on the evidence. #349 proved the properties the core needs directly: Cloudflare's unmodified DOFS schema over file-backed node:sqlite, surviving full restarts (including deletions and replacements), isolating workspaces by database path, refusing a newer on-disk schema rather than recreating it — at 110 MB and 0.08 s per operation, one process, no supervision surface. The declarative components map onto primitives already in that layer (<Dir>→mkdir, <File>→writeFile/readFile, <Branch>/<Commit>→DOFS's own git support, which is what Cloudflare's git shell command shims). #351 confirms a host capability and an imperative backend observe the same committed state in both directions. Per the spike's scope, no components were implemented. The FUSE caveat from #349 bears only on arbitrary native subprocesses, which xmd workflow does not require.

2. Which imperative capabilities can safely layer on top?

• Worker Shell — include initially, scoped. Runs natively, reuses Cloudflare's adapter byte-identical, and its containment was measured rather than assumed: ~50 escape attempts, zero leaks, and no native-execution path exists. Two conditions: run it inside a Deno Worker so a CPU-bound script cannot starve the host (measured: 6.4 s with the abort timer never firing), and describe it accurately as a workspace-scoped interpreter with 83 built-ins — never as native command execution.
• Worker JavaScript — defer. The full contract works, including an in-memory module graph with no materialization, live streaming, and capability access to DOFS. But isolation holds only in a compiled artifact (a deno run host lets a "locked" worker import jsr:/npm:), and CPU-bound user code cannot be preempted at all. As an optional extension over a declarative core, the cost of shipping it now exceeds its value; revisit if Deno adds a CPU/wall budget on new Worker(...), or if an out-of-process worker's startup cost is acceptable.

Is bundled workerd still necessary? Not for this scope. #347's decisive advantage was running Cloudflare's real backends; #351 shows both execution models also run natively in Deno over the same workspace — 139.7 MB carrying both backends plus DOFS versus 191 MB for the workerd host, operations in tens of milliseconds rather than tenths of a second, and no child process to supervise. workerd remains the only path to Cloudflare Containers, the hosted Workers surface (KV, R2, Queues, Durable Object identity), and guaranteed deployed-Cloudflare fidelity; if those enter scope, #347's evidence stands ready.

Recommendation (evidence, not decision): adopt Deno-local DOFS for the declarative durable core; include Worker Shell initially, scoped as above; defer Worker JavaScript; omit bundled workerd from the local topology, keeping #347 documented for hosted fidelity or Containers. All three PRs (#348, #350, #353) carry reproducible proofs whichever way you decide.