Repository navigation
Design KV Object Vector Engines Phase 8
Status: design (all three engines) + all three engine slices merged —
Phase 8 shadow-complete (2026-07-05): the KV/state slice
(ehdb#244, e2547200; worker KV shadow
worker#158, 7876be06), the
object/blob slice (ehdb#245,
e019ad9; worker object shadow worker#159,
1a24388, v5.59.0), and the vector slice
(ehdb#246, f6c5a6f; worker vector shadow
worker#160, c7b8872 / v5.60.0
0f57625). No tier is cut over to primary — all three engines ship
disabled-by-default as shadows only (KV NOETL_EHDB_KV, object NOETL_EHDB_OBJECT,
vector NOETL_EHDB_VECTOR, all default off). Phase 8 stays in progress overall
because per-tier primary cutover is the separately-gated Roadmap Phase 9
step (five independent cutovers: event-log, projection, KV, object, vector).
Tracks: noetl/ehdb#241 (completion program) · RFC (decided) · Roadmap Phase 8 · builds on Design: Event-Log Core Engine (Phase 6) + Design: Projection / Read-Model Engine (Phase 7).
Phases 6–7 put EHDB underneath NoETL's event log and on top of its tail as the read-model builder. Phase 8 takes over the three remaining internal platform storage tiers: the NATS-KV platform-state tier, the external object store platform-artifact tier, and the Qdrant platform-vector tier. Each is a separate engine that mirrors the Phase 6/7 driver + disabled-by-default shadow shape, so the tier is driver-selectable back to its incumbent (Roadmap Phase 10) and the cutover to serving from EHDB stays a later, separately-gated step (Phase 9).
Platform state only — never business data. All three engines store NoETL's own platform tiers: coherence/breaker state (KV), platform artifacts and Arrow-IPC state shards (object/blob), and platform RAG/catalog embeddings (vector). Tenant/domain KV, tenant object buckets, and business vector collections stay in external systems reached by playbook connectors and never move — that is the hard boundary the RFC draws and Phase 8 keeps.
These tiers never author the event log. KV entries, objects, and vectors are
derived state, artifacts, and indexes — not events. None of these engines writes
a noetl.event; event authorship stays with the gateway/server producer path
(unchanged since Phase 6). Each engine is structurally asserted to hold no
NoETL-event writer.
Loose server↔EHDB coupling. Server/gateway stay control-plane-only and stateless; the worker/system data-plane roles embed the engines in-process. A control-plane role that reaches any of these data-plane engines is refused by the guard before an engine opens — the same boundary the event-log/projection shadows enforce.
Confirmed by a source survey of the worker (fresh clone of noetl/worker main).
Business-data uses of the same stores (via playbook connectors) are out of scope
and must stay external.
| Internal keyspace | Where | Value | Access | Classification |
|---|---|---|---|---|
noetl_subscription_circuit bucket, key circuit.<subscription_id>
|
worker src/spool_runtime.rs (RFC #90 Phase 4) |
JSON CircuitRegistry snapshot |
get (rehydrate on restart) + put (persist each breaker transition); history=1; no TTL; no CAS |
platform — the worker's own subscription circuit-breaker resilience state |
| ChainHeads / ExecDescriptor coherence keys (#115) | program-scale multi-replica coherence | chain head / execution descriptor records | last-write-wins; multi-replica reads | platform — folds into the same tier as it moves off NATS-KV |
The concrete worker-owned bucket is a plain get/put store with history=1
(latest-revision) and no TTL/CAS in use today; the KV engine's interface is the
superset (get/put/delete/scan/CAS/TTL) so future platform KV keyspaces
(coherence, cursors) fold in without a new tier. Note: the #115 ChainHeads /
ExecDescriptor keys live in server-side code paths (outside this session's repo
scope); the worker's only in-repo NATS-KV bucket is noetl_subscription_circuit.
The #115/#166 WAL index and Feather state-shards are not NATS-KV — they are the
in-memory WAL index and the object tier below.
| Internal artifact | Key scheme | Format | Access | TTL/GC |
|---|---|---|---|---|
State shard (#166) |
noetl/env=…/region=…/cell=…/shard=s<NNNN>/…/execution=<eid>/state/<open|sealed>.feather |
Arrow IPC (Feather) slim chain |
object_put (throttled, state_materializer.rs) / object_get (cold-load on drive miss, state_reader.rs) |
open-shard idle TTL + max-open ceiling (writer-side) |
Result tier (#104) |
noetl/…/execution=<eid>/results/<step>/<frame>/<row>/<attempt>.<feather|json> |
Arrow Feather (tabular) or JSON |
object_put / object_get (result_materializer.rs, result_resolver.rs) |
server-managed; derivable from WAL |
Both are content-derivable platform artifacts (rebuildable from the WAL), and —
critically — the worker already reaches them through the server's
/api/internal/objects/{key} API (src/client/control_plane.rs), honoring the
data-access boundary.
The EHDB object engine therefore sits underneath the server's object endpoint (or
behind the worker's object client as a selectable driver), not as a new direct-store
path.
Platform RAG/retrieval vectors (playbook/runtime-surface docs, chunks, embeddings,
vector-index metadata). The worker has no external Qdrant client — RAG retrieval
already runs in-process via the EHDB Phase-E path (src/ehdb/rag.rs over
ehdb-reference's bounded retrieval helper, ingest/retrieve, top-k/byte/time caps,
data-plane-guarded). So the Phase-8 vector slice is largely about formalizing a
VectorDriver interface + shadow-parity harness over the engine that already
exists, not about removing a live external dependency from the worker. Any residual
external Qdrant usage for platform vectors (e.g. server-side catalog embeddings)
becomes the shadow's parity ground truth.
Each engine mirrors the Phase 6/7 driver + shadow pattern: a *Driver trait so the
tier is selectable (Phase 10), a LocalReference* EHDB implementation over the
bounded append-only stream/object primitives, secret-free views, bounded ops, and a
pure compare_*_parity for the worker's disabled-by-default shadow.
Interface (ehdb-reference::kv, ehdb#244):
pub trait KvStateDriver {
fn driver_name(&self) -> &'static str;
fn put(&self, req: &KvPutRequest) -> Result<KvPutOutcome>; // + CAS + TTL
fn get(&self, req: &KvGetRequest) -> Result<KvGetOutcome>; // TTL-aware
fn delete(&self, req: &KvDeleteRequest) -> Result<KvDeleteOutcome>;
fn scan(&self, req: &KvScanRequest) -> Result<KvScanOutcome>; // bucket + prefix
}Storage over EHDB primitives. Each write is one append to a single canonical
noetl_kv_state stream, scoped by a per-key subject
noetl.kv.<bucket>.<sha256hex(key)>.
A get is the latest record of that key's subject-filtered replay; a bucket scan
is the noetl.kv.<bucket>.> replay folded to the latest record per key. Keys
carrying .// (real NoETL keys like circuit.12345) are reduced to a
fixed-width SHA-256 digest token (the bucket stays a literal token, so the
bucket scan is unchanged); the original key rides in the record envelope so scans
reconstruct real keys and per-key reads filter the replay to the exact key.
KeepAll retention keeps the whole write history, so any past state is a replay
(append-only, immutable, replay-is-truth).
Semantics vs NATS-KV.
| Property | NATS-KV | EHDB KV engine |
|---|---|---|
| Read | latest revision (history=1) |
latest record of the key's subject replay |
| Delete |
delete/purge
|
tombstone record; subsequent get = absent (idempotent) |
| CAS |
update with expected revision |
put with KvCasExpectation::{Absent, Version(n)} → distinct conflict outcome, never appends on conflict |
| TTL | per-key TTL |
clock-free — caller supplies absolute expires_at_ms at write + now_ms at read; a read past expiry is absent |
| Version | KV revision | monotonic per-key version (advances across tombstones) |
Bounded + secret-free: 1 MiB value cap (over-cap ⇒ rejected), 4096-entry scan ceiling, no bucket/key/value in any outcome that reaches a metric.
Shadow parity — compare_kv_parity. Given the authoritative NATS-KV view of a
key and the EHDB get result, it checks presence (both present/absent),
value (byte-equal when both present), and TTL (equal expiry when both
present), with a single divergence reason. This is the dual-write proof: the worker
writes the authoritative NATS-KV path unchanged, also writes the EHDB engine, then
compares — never serving a read from EHDB.
Interface (ehdb-reference::object, ehdb#245):
pub trait ObjectBlobDriver {
fn driver_name(&self) -> &'static str;
fn put(&self, req: &ObjectPutRequest) -> Result<ObjectPutOutcome>; // content-addressed
fn get(&self, req: &ObjectGetRequest) -> Result<ObjectGetOutcome>; // digest-verified
fn list(&self, req: &ObjectListRequest) -> Result<ObjectListOutcome>;// prefix scan
fn delete(&self, req: &ObjectDeleteRequest) -> Result<ObjectDeleteOutcome>;
fn locate(&self, req: &ObjectLocateRequest) -> Result<ObjectLocateOutcome>; // presign-equivalent handle
}Storage over EHDB primitives. The engine stores the blob bytes once in a
content-addressed [LocalObjectStore] (ehdb-storage ImmutableObjectStore +
ObjectDigest::sha256) at objects/sha256/<hex>, and records a logical
key → digest mapping as one append to a single canonical noetl_object_store
registry stream, scoped by a per-key subject noetl.obj.<sha256hex(key)>. The
external key scheme (noetl/env=…/execution=…/state|results/…) rides in the record
envelope verbatim — it never touches the physical object path (only the digest
does), so keys carrying . / / / = round-trip (and =, which is not a safe
ObjectPath character, is sidestepped entirely). KeepAll retention keeps the whole
registry history, so replay is truth.
Subject-length defect fixed (ehdb#256 / ehdb#257, 2026-07-07). The subject originally hex-encoded the full logical key (
noetl.obj.<hex(key)>, 2 chars per key byte).ehdb-stream::Subject::newcaps a subject at 256 chars, so any key past ~123 bytes overflowed →InvalidIdentifier. Real platform coordinate keys run ~140-150 bytes (noetl/env=…/region=…/cell=…/shard=…/execution=…/state|results/…), so the object tier could persist no real state-shard (#166) or result-tier (#104) artifact — proven live on the worker v5.68.0 shadow drive, where every realobject_putwas rejectedoutcome=invalid. The fix (above) makes the subject a fixed-width SHA-256 digest token (noetl.obj.<sha256hex(key)>, a constant 74 chars) — the full key stays in the record payload, soget/list/locateresolve the real key without ever reversing the subject;latest_envelopefilters the replay to the exact key for digest-collision safety. Carried into the worker at v5.68.1 (ehdb-reference pin →bbc5047), and re-proven in-kind on the worker v5.69.0 shadow drive (outcome="invalid"→"mirrored", subject a 74-char digest). KV and vector carried the same latent hex-of-id pattern and are now fixed the same way (ehdb#259 / ehdb#260, 2026-07-08) — see the KV and vector sections above/below for the digest-token subjects, and the callout under Subject digest tokens (all three tiers) below.
Semantics vs external store.
| Property | External store | EHDB object engine |
|---|---|---|
| Write |
object_put (overwrite) |
append registry record → monotonic per-key version; bytes stored once (identical content dedups) |
| Read | object_get |
latest live registry record → get_verified (length + SHA-256) over the content-addressed blob; reports verified (retrievability), not the bytes |
| Delete | object_delete |
tombstone registry record (blob left for other referrers; physical blob GC is separate) |
| List | prefix scan | bounded noetl.obj.> replay folded to latest per key, prefix-filtered, ordered by key |
| Locate | signed external URL | in-cluster ehdb-object://<ns>/objects/sha256/<hex> URI (the presign-equivalent — EHDB is in-cluster) |
Object writes are already server-mediated (/api/internal/objects/{key}), so the
engine sits under that endpoint.
Bounded + secret-free: 16 MiB blob cap (over-cap ⇒ rejected), 4096-entry list ceiling, no key/bytes in any outcome that reaches a metric (the digest is a hash).
Shadow parity — compare_object_parity. Given the authoritative external-store
view (digest, byte_len) and the EHDB get result, it checks presence,
digest (byte-equal, exact because artifacts are content-addressed), byte
length, and retrievability (the EHDB blob passed get_verified), with a single
divergence reason. The worker writes the authoritative external path unchanged, also
writes the EHDB engine, then compares — never serving a read from EHDB.
Config: NOETL_EHDB_OBJECT = off | shadow | primary (default off);
primary is recognised but inert (compile-time PRIMARY_SERVE_ACTIVATED = false).
Interface (ehdb-reference::vector, ehdb#246):
pub trait VectorDriver {
fn driver_name(&self) -> &'static str;
fn upsert(&self, req: &VectorUpsertRequest) -> Result<VectorUpsertOutcome>;
fn query(&self, req: &VectorQueryRequest) -> Result<VectorQueryOutcome>; // top-k
fn delete(&self, req: &VectorDeleteRequest) -> Result<VectorDeleteOutcome>;
}Storage over EHDB primitives. Each upsert is one append to a single canonical
noetl_vector_index stream, scoped by a per-point subject
noetl.vec.<sha256hex(collection)>.<sha256hex(point_id)>. A collection query is the
noetl.vec.<sha256hex(collection)>.> replay folded to the latest live record per
point (the collection filter digests the collection identically, so it stays
consistent with the per-point subject). The collection + point id ride in the record
envelope verbatim (digested into subject tokens only), so ids carrying . / /
round-trip and a long id can never overflow the 256-char subject cap; the fold
guards on the exact collection for digest-collision safety. KeepAll retention keeps
the whole write history, so replay is truth. Query scoring is the same cosine +
descending-rank shape as ehdb_retrieval::InMemoryRetrievalCatalog::search_similar
(the Phase-E retrieval path), so a shadow's top-k tracks the incumbent retrieval —
this slice formalizes the in-process Phase-E retrieval as a first-class
VectorDriver rather than removing a live worker dependency (the worker has no
external Qdrant client).
Semantics vs Qdrant.
| Property | Qdrant | EHDB vector engine |
|---|---|---|
| Upsert | points + payload (overwrite) | append record → monotonic per-point version; latest wins |
| Query | top-k by cosine | bounded cosine top-k over the collection's live points, filtered to the query's model + dimensionality, ranked descending (ties broken by point id) |
| Delete | by point / collection | tombstone record; subsequent query drops the point (idempotent) |
Bounded + secret-free: MAX_VECTOR_DIMENSIONS (4096), MAX_VECTOR_QUERY_TOP_K
(64), MAX_VECTOR_PAYLOAD_BYTES (16 KiB). Over-cap ⇒ rejected; empty / non-finite
/ zero-norm vector ⇒ invalid. No collection / point id / vector / payload reaches a
metric.
Shadow parity — compare_vector_parity. Given the authoritative Qdrant top-k
and the EHDB query result for the same query, it checks id-set parity (same
top-k point ids), rank-order parity (same ordered id sequence), and score
monotonicity (the EHDB ranking is non-increasing within a tolerance, since float
scores differ across engines), with a single divergence reason. The worker writes the
authoritative Qdrant path unchanged, also writes the EHDB engine, then compares —
never serving retrieval from EHDB.
Config: NOETL_EHDB_VECTOR = off | shadow | primary (default off); primary
is recognised but inert (compile-time PRIMARY_SERVE_ACTIVATED = false).
- KV/state (merged, ehdb#244) — smallest, most self-contained tier (one concrete worker bucket; plain get/put with a clean CAS/TTL superset), so it landed the Phase-8 driver+shadow pattern for the other two to copy.
-
Object/blob (merged, ehdb#245) —
EHDB already owns an immutable object store and the keys are content-addressed, so
the parity check is a digest equality — a low-risk dual-write. Landed second
because the artifact tier is the highest-volume platform tier and benefits most
from EHDB co-location (state-shard cold-load,
#166). - Vector (merged, ehdb#246) — the engine already runs in-process (Phase E), so this was the least urgent: a driver-interface + shadow-harness formalization rather than a live-dependency cutover. Landed last, once the KV/object slices had proven the Phase-8 shadow shape end-to-end. Closes the Phase-8 engine set at shadow-complete.
Primary cutover of any tier is out of scope for every Phase-8 slice — it is the Phase-9 gated step (dual-run verify + documented per-tier rollback, kind before GKE).
| Env var | Default | Meaning |
|---|---|---|
NOETL_EHDB_KV |
off |
KV driver selection: off / shadow / primary. Unrecognised ⇒ off. |
NOETL_EHDB_OBJECT |
off |
Object/blob driver selection: off / shadow / primary. Unrecognised ⇒ off. |
NOETL_EHDB_VECTOR |
off |
Vector driver selection: off / shadow / primary. Unrecognised ⇒ off. |
NOETL_EHDB_VECTOR_MAX_DIMENSIONS |
2048 |
Worker-side embedding dimensionality clamp (crate ceiling 4096). |
Each respects the existing EHDB contract (NOETL_EHDB_ENABLED +
NOETL_EHDB_MODE=local_reference + data-plane NOETL_EHDB_CLIENT_ROLE +
NOETL_EHDB_LOCAL_REFERENCE_LOG). primary is recognised but inert in every
slice — a compile-time PRIMARY_SERVE_ACTIVATED = false per tier makes serving
structurally impossible until the Phase-9 cutover.
All three engines ship disabled by default and follow the same three-mode contract as the event-log/projection shadows:
-
off — strict no-op. No engine opened, no metric recorded;
/metricsand behaviour byte-identical to a build without the wiring. -
shadow — every authoritative write is also applied to the EHDB engine and
the two are compared (
compare_kv_parity/ digest parity / top-k parity). Reads are never served from EHDB; the incumbent path is untouched. - primary — recognised but refused with a distinct outcome; the compile-time guard makes serving structurally impossible this phase.
Observability: secret-free noetl_ehdb_kv_* / _object_* / _vector_* metric families
(op-family shape, labels operation + outcome only — never a bucket, key, value,
object path, or query text). Disabled ⇒ no lines rendered.
ehdb-selfcheck (worker-side, shipped in the worker image) carries the KV shadow
verbs (mirror-kv / kv-suite), the object shadow verbs (mirror-object /
object-suite), and the vector shadow verbs (mirror-vector / vector-suite) with
the standard exit-code contract (0 ok / 3 rejected /
4 guard-refused|invalid|primary-unavailable / 5 unavailable|parity-mismatch), for
in-image kind validation, mirroring the event-log/projection selfcheck verbs.
Standing validation evidence — KV slice (2026-07-05, no image build — the
~110-min worker image build evicts the session, so the in-container kind run is
DEFERRED as in Phase 6/7): cargo test ehdb-reference → 91 passed (18 KV engine
tests); cargo test ehdb::kv → 13 passed; clippy -D warnings clean; built
ehdb-selfcheck proves off = disabled no-op + empty metrics (exit 0),
shadow mirror-kv = mirrored with presence/value/TTL parity (exit 0), kv-suite =
7/7 engine capabilities + secret-free metrics (exit 0), control-plane role =
guard_refused (exit 4), primary = primary_unavailable inert (exit 4).
Standing validation evidence — object slice (2026-07-05, no image build, same
DEFERRED in-container kind run): cargo test -p ehdb-reference → 109 passed (18 new
object engine tests: put/get/list/delete/locate/dedup/scope-isolation/special-key
round-trip/oversize-reject/digest-verify/replay/parity); cargo test ehdb::object →
13 passed; clippy -D warnings clean; built ehdb-selfcheck proves off = disabled
no-op + empty metrics (exit 0), shadow mirror-object = mirrored with
digest/length/retrievability parity (exit 0), object-suite = 6/6 engine capabilities
(put/get/dedup/list/locate/delete) + secret-free metrics (exit 0), control-plane role
= guard_refused (exit 4), primary = primary_unavailable inert (exit 4), oversize
= rejected (exit 3).
Standing validation evidence — vector slice (2026-07-05, no image build, same
DEFERRED in-container kind run): cargo test -p ehdb-reference vector → 17 vector
engine tests (upsert/query-topk/delete/overwrite/top-k-truncation/over-limit-reject/
scope-isolation/model+dim filter/special-id round-trip/empty/replay/parity);
cargo test ehdb::vector → 14 passed; cargo test ehdb::metrics → 2 passed;
clippy -D warnings + fmt clean; built ehdb-selfcheck proves off = disabled
no-op + empty metrics (exit 0), shadow mirror-vector = mirrored with
id-set/rank-order/monotonicity parity (exit 0), vector-suite = 4/4 engine
capabilities (upsert/query-parity/top-k-truncate/delete) + secret-free metrics
(exit 0), control-plane role = guard_refused (exit 4), primary =
primary_unavailable inert (exit 4), over-limit dimensionality = rejected (exit 3),
zero vector = invalid (exit 4).
-
KV keyspace breadth. Only one concrete worker-owned NATS-KV bucket
(
noetl_subscription_circuit) is in the worker repo; the#115ChainHeads / ExecDescriptor coherence keys live in server-side paths (outside this session's scope). The KV engine's interface is the superset so those fold in without a tier change; their exact keyspace/value shapes are confirmed when the server side moves. -
KV values are UTF-8. The engine value type is
String(the circuit snapshot isserde_json::to_vec→ valid UTF-8). A future non-UTF-8 platform KV value hex/base64 encodes at the caller layer, matching the event-log engine'spayload: String. -
Object engine sits under the server object endpoint. The worker already reaches
the object tier via
/api/internal/objects/{key}, so the object driver is selected behind that endpoint (or the worker's object client), not a new direct-store path — preserving the data-access boundary. - Vector is already in-process. The worker has no external Qdrant client; the vector slice formalizes a driver over the existing Phase-E retrieval rather than cutting a live dependency. Server-side catalog-embedding Qdrant usage (if any) becomes the shadow's parity ground truth.
-
TTL/versioning shape. NATS-KV
history=1+ no TTL is the incumbent's actual usage; the engine offers monotonic versioning + clock-free TTL as a superset so richer platform keyspaces (leases, cursors) fold in without a redesign.
All three Phase-8 engines (KV, object, vector) are now built + shadow-complete. What remains is not more Phase-8 engine work — it is the Phase-9 cutover and the Phase-10 tunable surface:
-
Primary cutover of each tier — the separately-gated Phase-9 step. Five
independent per-tier cutovers, each
shadow→primary(EHDB serves reads, the incumbent is retired for that tier), dual-run verified against shadow parity history, per-tier rollback documented, kind-validated before GKE:-
Event log (
NOETL_EHDB_EVENTLOG) off NATS JetStream + Postgresnoetl.event. -
Projection / read-model (
NOETL_EHDB_PROJECTION) off the Postgres materializer. -
KV / state (
NOETL_EHDB_KV) off NATS KV. -
Object / blob (
NOETL_EHDB_OBJECT) off the external object store. -
Vector (
NOETL_EHDB_VECTOR) off Qdrant. Each tier'sprimarymode is inert today (PRIMARY_SERVE_ACTIVATED = false); activating it is Phase-9 work, not a config flip.
-
Event log (
- Production KV + object + vector store format (segmented + indexed) + compaction, plus physical content-addressed blob GC for tombstoned object keys, for the primary path.
- NATS-KV / external-object-store / Qdrant
*Driverimplementations for the tunable surface (Phase 10) so dual-run can select either engine per tier.
All three registry-backed tiers address a record's NATS subject by a fixed-width
SHA-256 digest token of the logical id, not the id's own hex. This is the single
shared invariant that keeps every per-id subject bounded under the 256-char
ehdb-stream::Subject::new cap regardless of id length:
| Tier | Subject | Digested token(s) | Literal token(s) |
|---|---|---|---|
| Object | noetl.obj.<sha256hex(key)> |
key | — |
| KV | noetl.kv.<bucket>.<sha256hex(key)> |
key | bucket (so noetl.kv.<bucket>.> scan is unchanged) |
| Vector | noetl.vec.<sha256hex(collection)>.<sha256hex(point_id)> |
collection, point id | — |
Why a digest, not the id's hex: a real ~140-150-byte platform coordinate key
hex-encodes (2 chars/byte) past the 256-char cap and Subject::new rejects it —
proven live on the object tier (worker v5.68.0 shadow, outcome="invalid"). A
64-char digest keeps each per-id token constant-width. The full ids always live in
the record payload (ObjectEnvelope.key, KvEnvelope.key,
VectorEnvelope.{collection,point_id}), so get/list/scan/query/locate
resolve the real ids without ever reversing the subject — the subject is purely a
routing token. Per-id reads filter the subject-matched replay to the exact stored
id, and the vector collection-scan fold guards on the exact collection, so a
(cryptographically infeasible) digest collision can never resolve or co-mingle a
colliding id's record.
Delivery: object ehdb#256 (worker
v5.68.1); KV + vector ehdb#259 /
ehdb#260 (worker pin bump
noetl/worker#172). The KV + vector fix
is forward-safety — the live KV path uses short circuit.<id> keys and there is no
live worker vector-upsert site, so nothing was broken in live use; the digest token
keeps a future long-key KV (#115 program-coherence) or platform-RAG vector-ingest
path from overflowing. Compatibility: moving to the digest subject moves where
records live; the affected data is shadow-tier EHDB platform state in throwaway kind
clusters (never business data, never GKE/prod, NOETL_EHDB_* flags default off), so
a clean cutover is fine and no migration is written.
- Design: Event-Log Core Engine (Phase 6) — the driver + shadow shape this design mirrors.
- Design: Projection / Read-Model Engine (Phase 7) — the second application of that shape.
- Roadmap — Phase 8 definition + Phases 9–10.
- RFC: Completion Program — the decided coupling + substrate shape (KV/object/vector tiers).
- Architecture — embedded roles + Rust-first worker integration.
- Home
- Architecture
- Architecture — the four engines
- Architecture — resilient KV core
- Consistency Invariants (per tier)
- Roadmap
- Sessions Log
- Claude Handoff
- RFC: Completion Program
- RFC: External EHDB Driver
- L1 Command-Bus Cutover (T4/T5 — prepared, human-gated)
- Prod Cutover — Event-Log Tier (Phase 9, Tier 1)
- Runbook: Async Event-Log Mirror
- Durable Event-Log — Prod Durability Sign-off (§C, slice 6)