Skip to content

History / Design Execution Chain Store

Revisions

  • docs: v0.4.5 — link-defined ordering, and the capability it removes noetl/ai-meta#362. The chain is ordered by following prev-links; event_id order is never consulted, so an event committing into the middle of an id-ordered read is a link rather than a shifted position. ⚠ Records the trade explicitly: v0.4.3/v0.4.4 could reconstruct a forked chain from log order and v0.4.5 refuses it instead. That reconstruction was the same id-order assumption that caused the false divergence, so removing it is the fix — but the store now covers fewer executions, and a source that refuses everything also reports zero divergence.

    @claude claude committed Sep 30, 2026
  • docs: CORRECTION — v0.4.4 is rolled back, and a core assumption of the design is false noetl/ai-meta#362. populate_from_log recomputes chain edges from log order and treats that order as a stable prefix that only grows at the tail. An event can commit into the MIDDLE of an ORDER BY event_id read, because a snowflake event_id is minted before the insert — so commit order is not id order. Every position after the interior insert then reads as a content conflict. Proven by position arithmetic: the store held at position 72 what the log holds at position 73 — exactly one missing interior row. ⚠ No available column is commit-ordered. created_at is stamped at MINT time, so it always agrees with event_id order and cannot detect the reordering; I ran that comparison as a refutation and it refuted nothing. v0.4.4 stands — both defects it fixed are real — but it was not what was breaking prod, and the earlier 5-in-79 was most likely this same mechanism.

    @claude claude committed Sep 30, 2026
  • docs: v0.4.4 — a behind snapshot is staleness, and the chain store is live in prod noetl/ai-meta#360. Design-Execution-Chain-Store + Sessions-Log + Home. New section "Staleness is not a conflict — and the same shape is a second writer", which is the design point of this release: the crate REPORTS `FromLog::StaleLog` as unresolved rather than deciding, because from a single snapshot benign staleness and a foreign writer are indistinguishable, and only the consumer can re-read the log to tell them apart. My first version decided it, and the noetl/ai-meta#358 regression test failed on the first run. Also recorded: a content conflict now revokes serve-trust (previously this function wrote nothing, so the still-matching coverage handed a `chain_if_authoritative` caller a chain just proved wrong); coverage is monotonic; and the monotonic-coverage mutant SURVIVED its first test, which populated 5 then read 2 and returned StaleLog before the guard was ever reached. ⚠ Corrections: the status line said "flag-gated, default OFF, not enabled anywhere" and the Home link said the store "cannot serve yet". It is ARMED IN PRODUCTION since 2026-09-29 17:30Z on server v3.117.2 — 772 comparisons, 0 failures. Version floor raised v0.4.3 -> v0.4.4, because v0.4.3 carries the false-divergence classification that rolled back the first ramp.

    @claude claude committed Sep 29, 2026
  • Design: the chain store shipped in v0.4.3 — record the version FLOOR The page said 'built, not enabled'. It is now on main and tagged, so the thing a reader needs is the VERSION FLOOR, not the build status: v0.4.3 or later, because v0.4.2 has the store and the watermark but neither populate_from_log nor the coverage record — and a reader on v0.4.2 therefore cannot tell a truncated partition from a whole one, which is the defect the page documents. Also records that FORMAT_VERSION is 1 across v0.3.0/v0.4.2/v0.4.3, so moving between them does not trip the on-disk layout gate — the question anyone bumping the pin will have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    @claude claude committed Sep 29, 2026
  • Design: populate_from_log + the coverage record; both defects closed The chain store's page said it could not serve, and named two defects. Both are now fixed and kind-proven, so the page has to say how rather than leave a reader with a stale blocker. Adds the section a reader needs most — populate_from_log and the COVERAGE record — and states plainly what the watermark cannot express: its first_seq is the STORE's sequence, 1 for any fresh partition regardless of where the execution began, which is exactly how a chain starting at an execution's third event read as authoritative. ⚠ Records that the root signal is deliberately NOT an event-type name. Only 62 of 595 executions start with playbook_started; 533 start with playbook.initialized, so a check built on the first guess would have been inert for 90% of them. The root is position 0 in the log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    @claude claude committed Sep 29, 2026
  • Design: the execution-partitioned chain store, and why it cannot serve yet New page for ehdb-l0's DurableChainStore + ChainPopulator (RFC noetl/ai-meta#355). Written from the diff and from the kind measurements, not from the RFC's summary. The load-bearing parts, recorded because each is a guard whose purpose someone would otherwise delete as noise: * the three states the watermark separates — `None` never means "no events", `Some(vec![])` does; * the ASYMMETRIC append/watermark ordering, and the defect that produced it (marking before every append left Authoritative{1,1} over zero events on the common mid-flight-arming path). v0.4.1 carries that bug; v0.4.2 is the first tag without it; * `apply_replicated` deliberately does NOT enforce the head, because async replication delivers out of order — found by a mutation test that could not construct a gap at all; * `chain_is_complete` compares against the partition SPAN, so it catches a hole in the middle and NOT a missing beginning — which is exactly why the open defect is undetectable from inside the store. ⚠⚠ Both measured defects are stated with their numbers and the shared cause (a chain edge stamped from an in-memory map that does not survive a restart), so nobody reads this page and concludes the store is ready to serve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

    @claude claude committed Sep 28, 2026