🔬 Spike: bundle workerd into the compiled XMD host (#347)

[taras]

Why

#347 asks for evidence: can Executable.md ship one compiled executable that carries a Cloudflare Worker and a platform workerd, launch that Worker locally, and keep a Durable Object Workspace across restarts? #346 needs that evidence to select, reject, or limit the bundled-local-host topology. This PR commits the reproducible spike and its findings.

What changes

Before: the repository had no Cloudflare artifacts and no answer to whether the bundled-local-host topology is feasible.

After: deno task spike:347 fetches a digest-pinned workerd, builds a 191 MB proof executable that embeds workerd + a Worker with one Durable Object + capnp configs, and runs a 7-scenario suite proving packaging, supervision, persistence, isolation, the @cloudflare/computer filesystem Workspace, and two of three execution backends under standalone workerd. Findings are written up in spikes/347-workerd/evidence/EVIDENCE.md.

How it works

deno task spike:347
  → fetch: npm tarball → vendor/workerd, sha256 vs manifest.ts
  → build: esbuild worker bundle → deno compile --include workerd,worker.js,configs → dist/proof
  → test: each scenario drives dist/proof, which materializes the embedded files into
    ~/.cache/xmd-spike-347/<version>/ (digest-checked), launches workerd on 127.0.0.1:0
    (--control-fd=1 reports readiness+port on stdout), performs HTTP ops against the
    bundled Worker, and tears the child down on scope exit

Every proof do invocation is a complete start/op/stop cycle, so consecutive invocations prove restart persistence by construction.

Key findings (full detail in EVIDENCE.md)

• Feasible. Bundle, materialize, supervise, restart — all work with no Deno, Node, or Wrangler in the environment (asserted with a stripped PATH and fresh HOME). Warm cycle 0.27 s; cold materialization 1.46 s.
• Persistence works via workerd's localDisk DO storage (the same mechanism miniflare uses) — but the schema marks it experimental, subject to backwards-incompatible change.
• Computer 0.1.1 filesystem Workspace works under standalone workerd (nodejs_compat, @platformatic/vfs as only loaded peer). Its SQLite schema refuses downgrades → exact pin mandatory.
• worker-shell and worker-javascript backends work, given a workerLoader binding, a WorkspaceServiceProxy re-export (ctx.exports loopback), and waitUntil wiring. Container backend also works through workerd's localDocker engine (Docker daemon required): computerd workload + egress-proxy sidecar created by workerd itself, bidirectional /workspace sync, and a fresh container restored from DO SQLite after restart — the exact Dockerfile/config/wiring that ran are committed under evidence/container/; workerd leaks its containers on SIGTERM (recorded).
• Supervision gaps recorded as contracts: SIGKILL of the host orphans workerd (no control-fd-close exit); @effectionx/process daemon teardown alone never terminated the child on natural exit under Deno — the host registers an explicit child SIGTERM ensure first.
• No sandbox: workerd explicitly disclaims hardened sandboxing; local execution has user-privilege blast radius.

What must stay true

• The four verification gates keep their exact scope — spikes is in the root deno.json exclude, the spike pins its own deno.lock, and no root dependency changed. Verified by running lint/check/test/check:jsr at root.
• Builds install nothing — the workerd download lives in the spike's fetch task, never in build.
• The production XMD contract is untouched — no packages/* file changed.

How to verify it

• deno task spike:347 from the repo root (macOS arm64 or any pinned platform; first run downloads ~32 MB from npm).
• counter survives full restarts… proves DO SQLite persistence + state-dir/identity isolation; fails if localDisk or identity derivation regresses.
• workspace filesystem frontier… proves the Computer Workspace frontier (incl. create/delete/create) across restarts; fails if the vfs schema or storage wiring breaks.
• worker-shell and worker-javascript backends… fails if the loader/proxy/waitUntil wiring regresses.
• Container backend: evidence/container/ holds the artifacts; reproducing needs Docker running, docker pull cloudflare/proxy-everything:main, and docker build -t probe-computerd.
• clean stop… / SIGKILL… assert the teardown contract both ways with child-pid liveness probes.

Scope

Included

• spikes/347-workerd/ (self-contained project), root task aliases, .gitignore entry for vendor/.

Intentionally unchanged

• No <Workspace> API, no provider, no topology selection — Define the local Workspace host topology #346 records the decision from this evidence; Implement retained Workflow Workspace restoration #218 moves only after that.
• No release-workflow changes: multi-platform bundling is measured (per-platform digests are already pinned in manifest.ts) but not wired into release.yml.

New dependencies

• Spike-local only (own deno.json + lock, invisible to the workspace): @cloudflare/computer@0.1.1, @platformatic/vfs@0.4.0, esbuild@0.28.1, plus the already-pinned effection stack. The workerd binary is fetched, digest-verified, and gitignored — not an npm dependency, deliberately (cross-compile deps:target fragility, see EVIDENCE §1).

Risks and limitations

• Worker-side worker/worker.mjs uses async/await — the Workers platform contract mandates it; host and tests are Effection generators.
• localDisk persistence format and Computer 0.1.1 are both explicitly unstable; every version is pinned and digest-checked, and upgrade behavior is documented in EVIDENCE §3-4.
• The spike suite runs only via deno task spike:347 (not in CI) — it downloads a binary and needs ~200 MB scratch, which CI jobs shouldn't pay per-push during a spike.

[github-actions]

PR #348: 🔬 Spike: bundle workerd into the compiled XMD host (#347)

20 files, +2079 / -2

Scope

🔴 PR has 2081 lines changed. Split into focused PRs.

🟡 2081 lines changed. PRs under 400 receive more thorough review.

🟡 PR mixes config and source changes.

Structural

🟡 7 console statements.

Slop

✅ Slop indicators look low.

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.

[taras]

We're not doing this anymore.