Skip to content

fix: durable volumes survive a redeploy - #10

Merged
apresmoi merged 20 commits into
mainfrom
fix/durable-volume-reattach
Sep 4, 2026
Merged

fix: durable volumes survive a redeploy#10
apresmoi merged 20 commits into
mainfrom
fix/durable-volume-reattach

Conversation

@apresmoi

@apresmoi apresmoi commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Durable volumes survive a redeploy

Fixes a P0 that already destroyed real data: a newsroom's entire message
history — reporters' story pitches, the editor's assignment — was lost to a
routine docker rm + recreate. Workspace kind: volume resources and durable
Moltnet sqlite stores got a brand-new empty Docker volume on every redeploy,
through every launch path.

The three original holes

  1. Run-scoped names. createPersistentVolumeName folded a freshly minted
    NOOPOLIS_RUN_ID into every durable volume name, and silently DISCARDED an
    author's explicit name whenever a run id was present.
  2. Bootstrap could not seed. runProject mounted with volume-nocopy, the
    exact flag that suppresses the copy-up delivering the image's
    .spawnfile-volume-init preimage that the Daimon ownership guard requires.
  3. Hand-launched containers got nothing, silently. Nothing failed closed
    when a durable path had no backing volume.

What changed

  • Workspace volumes, durable Moltnet stores and open-mode token directories are
    lifecycle: "exclusive-reattach", named from the plan root plus the
    deployment lineage exactly as durable memory banks already were. An
    author-declared name is used verbatim. createPersistentVolumeName now takes
    no name at all and serves only genuinely run-scoped mounts.
  • volume-nocopy dropped for compiler-owned persistent mounts. Copy-up only
    populates an EMPTY volume, so reattach is unaffected. src/target/* keeps its
    own.
  • require_backed_mount in the entrypoint and the Daimon root wrapper refuses
    to start when a durable path is not a real mount point, naming the mount id,
    path and volume. SPAWNFILE_ALLOW_EPHEMERAL_STATE=1 opts out. Comparison uses
    the kernel's octal-escaped mount-point form.
  • Declared names are unique-checked across EVERY mount source, and two
    resources whose declared names collapse onto one backing path are rejected.
  • Only an author-declared name is published in the distribution report
    (declared_volume_name) and honoured verbatim by a sourceless image
    deployment; a derived name never travels, since it encodes the creator's plan
    root.
  • spawnfile dev up gets its own lineage namespace, so a dev deployment can
    never resolve to a production deployment's derived volumes, and refuses to
    start on an author-declared name unless --allow-declared-volumes is passed.
  • spawnfile down --volumes never removes an author-declared volume; it reports
    them in the receipt's new optional skipped_volumes.

Known tradeoff

An organization declaring any of these resources loses the concurrent blue/green
canary path and must stop-and-reattach. This was already true for anything
declaring memory banks. Stated in specs/SPEC.md and specs/CONTAINERS.md.

Follow-ups, deliberately not in this branch

  • P2-2 daimon-organization-acceptance-store and daimon-tool-state-*
    remain run-scoped. After a redeploy the bridge holds receipts the runtime no
    longer knows about. Rated wider than originally scoped; worth its own change.
  • P2-D The declared-name refusal gates only the dev verb. spawnfile run
    and up --deployment scratch attach declared volumes with no gate. New in
    this branch (pre-branch, run run-scoped declared names so it could never
    reach production's volume). Documented in CONTAINERS.md, so not a defect, but
    the design is inconsistent and the rationale for refusing dev up applies
    identically.
  • P3-E The dev refusal lands after the full image build. The compile report
    exists before the Docker build; moving the check between them is the fix.
  • P3-F Compile-side declared names are unvalidated. name: "clank-store,readonly"
    emits --mount source=clank-store,readonly,... and Docker's CSV parser takes
    readonly as an option. The manifest schemas should mirror the consumer regex
    already enforced in distributionReportSchema.ts.
  • P3-H No migration note for operators who pinned NOOPOLIS_RUN_ID. Their
    old derived volumes are orphaned, not deleted, after upgrade.
  • containerArtifacts.ts is 424 lines, under the 454 it was but still over the
    repo's 400 ceiling, as are several pre-existing files.

Review

Two independent adversarial review rounds. Round one blocked on a cross-kind
declared-name collision that silently merged two states, a sourceless image
deployment ignoring declared names while the new spec text promised otherwise,
and a mount guard that false-positived on paths containing spaces. All three
fixed and re-verified. Round two found no surviving data-loss path and cleared
the branch to merge, with two call sites left unlocked; those are now
mutation-locked, along with down --volumes.

Every guarantee here is mutation-checked: reverting the naming fix, restoring
volume-nocopy, neutering the mount guard, collapsing the dev lineage,
bypassing the mount merge, or treating declared names as deployment-owned each
turns a test red. A test asserting prepare_volume_resource was merely called
passes throughout the original bug; only asserting that data survived a
docker rm catches it.

Cold rebuild: 3394 passed, 0 failed, 948 suites. Boundaries clean.

--- PULL REQUEST DESCRIPTION ---

# Durable volumes survive a redeploy

Fixes a P0 that already destroyed real data: a newsroom's entire message
history — reporters' story pitches, the editor's assignment — was lost to a
routine `docker rm` + recreate. Workspace `kind: volume` resources and durable
Moltnet sqlite stores got a brand-new empty Docker volume on every redeploy,
through every launch path.

## The three original holes

1. **Run-scoped names.** `createPersistentVolumeName` folded a freshly minted
   `NOOPOLIS_RUN_ID` into every durable volume name, and silently DISCARDED an
   author's explicit `name` whenever a run id was present.
2. **Bootstrap could not seed.** `runProject` mounted with `volume-nocopy`, the
   exact flag that suppresses the copy-up delivering the image's
   `.spawnfile-volume-init` preimage that the Daimon ownership guard requires.
3. **Hand-launched containers got nothing, silently.** Nothing failed closed
   when a durable path had no backing volume.

## What changed

- Workspace volumes, durable Moltnet stores and open-mode token directories are
  `lifecycle: "exclusive-reattach"`, named from the plan root plus the
  deployment lineage exactly as durable memory banks already were. An
  author-declared name is used verbatim. `createPersistentVolumeName` now takes
  no name at all and serves only genuinely run-scoped mounts.
- `volume-nocopy` dropped for compiler-owned persistent mounts. Copy-up only
  populates an EMPTY volume, so reattach is unaffected. `src/target/*` keeps its
  own.
- `require_backed_mount` in the entrypoint and the Daimon root wrapper refuses
  to start when a durable path is not a real mount point, naming the mount id,
  path and volume. `SPAWNFILE_ALLOW_EPHEMERAL_STATE=1` opts out. Comparison uses
  the kernel's octal-escaped mount-point form.
- Declared names are unique-checked across EVERY mount source, and two
  resources whose declared names collapse onto one backing path are rejected.
- Only an author-declared name is published in the distribution report
  (`declared_volume_name`) and honoured verbatim by a sourceless image
  deployment; a derived name never travels, since it encodes the creator's plan
  root.
- `spawnfile dev up` gets its own lineage namespace, so a dev deployment can
  never resolve to a production deployment's derived volumes, and refuses to
  start on an author-declared name unless `--allow-declared-volumes` is passed.
- `spawnfile down --volumes` never removes an author-declared volume; it reports
  them in the receipt's new optional `skipped_volumes`.

## Known tradeoff

An organization declaring any of these resources loses the concurrent blue/green
canary path and must stop-and-reattach. This was already true for anything
declaring memory banks. Stated in `specs/SPEC.md` and `specs/CONTAINERS.md`.

## Follow-ups, deliberately not in this branch

- **P2-2** `daimon-organization-acceptance-store` and `daimon-tool-state-*`
  remain run-scoped. After a redeploy the bridge holds receipts the runtime no
  longer knows about. Rated wider than originally scoped; worth its own change.
- **P2-D** The declared-name refusal gates only the `dev` verb. `spawnfile run`
  and `up --deployment scratch` attach declared volumes with no gate. New in
  this branch (pre-branch, `run` run-scoped declared names so it could never
  reach production's volume). Documented in CONTAINERS.md, so not a defect, but
  the design is inconsistent and the rationale for refusing `dev up` applies
  identically.
- **P3-E** The dev refusal lands after the full image build. The compile report
  exists before the Docker build; moving the check between them is the fix.
- **P3-F** Compile-side declared names are unvalidated. `name: "clank-store,readonly"`
  emits `--mount source=clank-store,readonly,...` and Docker's CSV parser takes
  `readonly` as an option. The manifest schemas should mirror the consumer regex
  already enforced in `distributionReportSchema.ts`.
- **P3-H** No migration note for operators who pinned `NOOPOLIS_RUN_ID`. Their
  old derived volumes are orphaned, not deleted, after upgrade.
- `containerArtifacts.ts` is 424 lines, under the 454 it was but still over the
  repo's 400 ceiling, as are several pre-existing files.
@apresmoi
apresmoi merged commit d7b02ee into main Sep 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant