Skip to content

host: atomic apply batch verb for local-store, per-call durability model #609

Description

@mfw78

From the local-store durability study (three-lens fable analysis, 2026-07-25). Decision recorded in ADR-0014 (#610).

Decision

The local store is per-call committed durability, not per-event atomic: each set/delete is an individually fsync-durable redb transaction, program-order, read-your-writes, and a trap (fault, panic, OutOfFuel, deadline, crash) freezes state at the last completed call and never rewinds it. Per-event atomicity is rejected: it would roll back the keeper Journal's RESERVED marker on exactly the trap it exists to survive, breaking the reserve/commit/reconcile design (#583-#587, #574), and it defends against nothing (dispatch is single-actor serialised, redb single-process). The sanctioned atomicity scope is a single opt-in batch verb.

API surface: no new host verbs beyond apply

The host primitive is complete: get/set/delete/list-keys/count/contains/len plus apply(set|delete). set is already an upsert (insert or overwrite). No compare-and-swap, atomic increment, or prefix-delete host verb is warranted, because every module is key-isolated by a keccak prefix (ADR-0003) and dispatch is single-actor serialised, so no two writers ever touch the same key: get-then-apply is effectively atomic, and prefix-delete composes as list-keys + apply(deletes). WIT variants stay additively extensible, so a conditional write-op can be added later if a concrete need appears; not now. Enrich the SDK, not the seam: every WIT addition is a versioned cross-repo change after the cleave, so a minimal host seam is itself the low-effort path.

Build

Host (Wave 1, done)

  • apply WIT verb + ModuleStore::apply (host: atomic apply batch verb for local-store #611): additive apply(list<write-op>) with write-op = set(key,value) | delete(key), one begin_write/commit, whole-batch net quota projection, op-count and byte caps, all-or-nothing. Landed.

SDK ergonomics (Wave 2, after #611 merges, needs apply in the bindings)

The pain is hand-serialising every list<u8> and rewriting get-modify-set and batch patterns; all pure SDK sugar over the primitives, zero host surface. A module should write typed, intention-revealing code and never touch raw bytes or hand-roll a batch.

  • TypedCell<T>: a borsh-typed key (cell.get/cell.set). The single biggest win; kills manual serialisation.
  • Typed keyed collection: list-by-prefix plus typed values, for sets of things (watches, orders), so modules stop hand-rolling indexes.
  • Counter / accumulator helper (get-inc-set, safe under single-actor dispatch).
  • WriteBatch builder over apply (stage in guest memory, flush in one host call; a pre-flush trap discards for free) and a clear_prefix helper (list-keys + apply-deletes, atomic).
  • Journal guard combinator: reserve, run the submit closure, then commit or park, enforcing reserve-before-effect ordering by API shape. The Journal's single-key markers stay on plain set (they span an await, so they structurally cannot ride one apply).

Test kit (Wave 1, done)

Constraints

apply is an additive nexum:host@0.1.0 verb landed pre-cleave (cheap in-monorepo fold, expensive cross-repo ripple after the carve). The Journal and the reserve/commit/reconcile code stay untouched; the SDK helpers add no host surface.

Acceptance

apply lands additive, quota-projected, capped, all-or-nothing (done, #611); the trap-injection harness proves crash convergence from every torn prefix (done, #612); ADR-0014 and the docs state the per-call contract with no rollback claim (#610, #606); the Wave 2 SDK helpers give module authors typed cells, collections, counters, atomic batches and a Journal guard, so no module reimplements store atomicity or serialisation by hand.

Metadata

Metadata

Assignees

No one assigned

    Labels

    component/engine-hosthost-interface implementations (the host impls layer)component/sdknexum-sdk / shepherd-sdk, proc macros, cargo-nexumneeds-designRequires thinking. No one has bandwidth.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions