v0.9.0

v0.9.0: the terminal stage, blobs in their place, and reads shaped to what they answer

Twelve changes since v0.8.0, on one theme: what a run costs the store and the engine to keep, read, and finish. Four reshape contracts and set this release's migration section.

• The terminal stage (#74, #76). A run's terminality commit replaces its whole nonterminal stage — input, operation history, cursor, failure, cancel — with one terminal record, in the same atomic step. What a finished run keeps is its identity, outcome, output, commit time, failure, cancel request, and failed unwinds; everything else is released at terminality, not at retention. bbolt's layout is rebuilt around it: one append-only active bucket keyed by run and tag with operations in resolution order, the cursor on its own, a terminal row per finished run, an expiry index so reap walks its victims and decodes nothing, and a staged bucket that frees released stages in batches. ListRuns leaves the store contract, InputBytes on a terminal run returns ErrRunTerminal, and every reserved proto field is deleted outright.

• Blobs in their place, and cached (#78, #81). An input, state, or output over half a page lives in a nested bucket beside its row, so writing or deleting it touches its own pages; smaller ones stay inside their records. Fat-state disk bytes drop about 65%. The store keeps in-memory copies of the blobs of runs in flight and of the large outputs of terminal runs, two bounded LRUs (64 MiB and 16 MiB by default) set through bbolt.Open options or the URI query. Input, State, and Output slices are declared immutable and shareable: stores retain and return shared memory, and bytes per op fall 65–75% in fat-input scenarios.

• Reads shaped to what they answer (#83, #85). GetRunHead returns a run without its blobs or history, and it is what Status, Wait, the lookups, await bookkeeping, and recovery read — polling many idle runs no longer churns the blob cache, and cold-run polling is about six times faster. The reconcile loop reads the full record once per dispatch and carries it; a cancel goes through the engine, which marks the run dirty so the loop re-reads. A straight run costs three store reads instead of eleven, and allocations per op drop 25–39% across the population scenarios. Wait registers its watcher before its single read.

• Cancel tightened (#85). A cancel request after the first is a true no-op. It used to re-preempt whatever attempt was running, including the retry the contract says continues cooperatively.

• The perf suite as an instrument (#75, #77, #79, #80, #82). Recovery and SupersedeCycle sized past a single leaf per bucket, a shape matrix of none, slim, and fat for input, state, and output, output scenarios that read their output, a cold working-set scenario, and reported polls/sec and reads/run.

Breaking changes and migration

• bbolt on-disk layout changed with no migration: a database written by v0.8.0 or earlier is not readable by this driver. Discard it.
• Terminal runs release their input: Run.InputBytes and the generated typed Input() return durable.ErrRunTerminal once the run is terminal. Read the input before Wait returns, or read the output, which the reducer folded it into.
• Immutable blobs: the slices InputBytes, OutputBytes, and a store's RunRecord return are shared with the store and must not be modified. The generated typed accessors still return caller-owned messages.
• Store SPI (store/driver): ListRuns is gone; GetRunHead is new and required; ListNonterminal returns heads; RunRecord.CompactTerminal is the shared terminal-stage rule; Clone shares blob slices. store/mem keeps Runs() as a test enumeration.
• bbolt driver: Open(path, opts ...Option) with WithBlobCache and WithOutputCache; URI query keys blob_cache and output_cache with K/M/G/KiB/MiB/GiB units, 0 disabling; the database is opened with NoFreelistSync.
• Storage proto: RunMeta drops its input, Terminal is new, Cursor.awaiting_run_id and every reserved field are removed; buf's RESERVED_MESSAGE_NO_DELETE and FIELD_NO_DELETE_UNLESS_NUMBER_RESERVED are excepted for the internal schema.

contrib/durableotel v0.9.0 is released in lockstep.

Modules

• github.com/dangra/durable@v0.9.0
• github.com/dangra/durable/contrib/durableotel@v0.9.0

What's Changed

• perf: size Recovery and SupersedeCycle past a single leaf per bucket by @dangra in #75
• Terminal stage: release input and step states when a run ends by @dangra in #74
• proto: drop every reserved and deprecated field by @dangra in #76
• perf: fat state and fat output scenarios by @dangra in #77
• perf: the shape matrix by @dangra in #79
• perf: drop the FatState and FatOutput composites by @dangra in #80
• bbolt: large inputs, states, and outputs in buckets of their own by @dangra in #78
• perf: the output shapes read their output after Wait by @dangra in #82
• bbolt: cache the blobs of the runs in flight; Input, State, and Output are immutable and shareable by @dangra in #81
• store: GetRunHead, the read for observation by @dangra in #83
• Drop the perf.test binary committed by mistake by @dangra in #84
• engine: the reconcile loop reads the full record once per dispatch by @dangra in #85
• Release v0.9.0 by @dangra in #86

Full Changelog: v0.8.0...v0.9.0