v0.8.0 — incremental state-fold strategy (config-selected)

What's new

A second, config-selected state-fold strategy: incremental (PR #26, #25).

The reference fold (DecodeState) is a stateless pure reducer: every ledger it rebuilds its whole in-memory mirror from the prior LedgerState and re-sorts the full typed state — O(total accumulated state) per ledger, so replay throughput decays as users accumulate. Measured on a production mainnet replay: ~312 ledgers/s with an empty prior vs ~51 ledgers/s carrying ~600k ledgers of accumulated state.

blend.Config gains a StateMode knob, selected once at construction and swapped as a whole strategy:

• paranoid (default, and the default when the knob is empty) — today's code path, byte-for-byte. It remains the reference oracle: fully stateless, run-twice byte-identical, the strategy every optimization is validated against.
• incremental — a persistent builder mirror carried across ledgers plus caches for the per-user position blocks and their global sort order. Per-ledger cost drops to O(this ledger's changes) plus output materialization. Output is byte-identical to paranoid — same slices, same ordering, same serialized checksums — enforced by a permanent CI parity gate (golden fixtures + a synthetic multi-ledger churn sequence, per-ledger state and delta-stream comparison, and a red-check that proves seeded divergences are caught).

Measured (real mainnet checkpoint state, 20,000 real ledgers folded from a stored production checkpoint at ledger 57,166,999 carrying ~600k ledgers of accumulated state):

• Byte-parity on production data: sha256 state checksums identical between modes at every measured interval (ledgers 57,171,999 / 57,176,999 / 57,181,999 / 57,186,999).
• Fold throughput at real state size: paranoid 372.1s for 20k ledgers (53.8 l/s — independently reproducing the production replay's observed ~51 l/s decay) vs incremental 4.8s (4,198.9 l/s) — 78.1x. The fold stops being the replay bottleneck; stateless extraction becomes the dominant term.

In-repo benchmark at 50,000 carried user positions, one touched user per ledger: paranoid 453.6 ms/ledger vs incremental 10.4 ms/ledger (~44x).

Statefulness contract

An incremental adapter carries fold state between calls: do not share one across concurrent folds, and treat the returned LedgerState as immutable. DecodeState remains a pure function of (prior content, changes): the carried mirror is only trusted when prior is the adapter's own previous return value; any other prior (first ledger, checkpoint restore, rewind) reseeds from it. The default paranoid mode is untouched and fully stateless.

Compatibility

Minor version bump: additive API only (StateMode on blend.Config). No LedgerState shape change — snapshots and fold checkpoints round-trip unchanged between versions and between modes; a fold may be seeded from a checkpoint written by either mode. Paranoid remains the default and the reference; consumers opt into incremental deliberately, per deployment.

Full diff: v0.7.1...v0.8.0