Repository navigation
RFC Server EHDB Coupling and Storage Substrate
Status: DECIDED — applying (2026-07-05). This is not a review gate; it records decisions being implemented. Tracks: noetl/ehdb#241 (completion program) · noetl/ehdb#234 (integration umbrella).
This RFC records the decided shape for two things: how EHDB couples to the NoETL server tier (loose), and what EHDB becomes for the NoETL platform (its self-sufficient internal storage fabric). It is grounded against the current Architecture and Roadmap pages — Phase 5 integration (A–E) is complete (2026-07-05); the roadmap ahead is restructured by this RFC into the completion trajectory below.
The boundary the wiki already enforces is unchanged:
gateway = gatekeeper
worker = atomic compute
playbook = ephemeral blueprint
shared cache = state vehicle
event log = source of truth
NoETL is event-sourced: the append-only event log is the source of truth and platform state is a materialized projection of it. Today that log rides NATS JetStream and is materialized into PostgreSQL by a server/worker materializer, with the derived read-model in Postgres. That path is the platform's scaling pressure point — the off-server state-builder work, the unbounded in-memory WAL index (noetl/ai-meta#166), the result-tier / materializer soak (noetl/ai-meta#104), and the sharded-state-builder RFC all circle the same limit: the JetStream→Postgres-materializer log-and-projection path does not scale cleanly with execution volume and multi-replica affinity.
EHDB exists to solve that. It is a distributed event-log engine plus projection engine purpose-built for NoETL's event-sourcing shape: it writes, orders, persists, and serves the event log, and builds and serves the read-model projections off it — replacing the JetStream+Postgres-materializer path with one durable fabric designed for the workload.
EHDB is NoETL's platform storage fabric — a family of core engines under one umbrella, each fitted to a workload type:
| Core engine | Serves | Replaces (NoETL-internal) |
|---|---|---|
| Event-log engine | Append, order, persist, serve the append-only noetl.event log |
NATS JetStream (log transport) + Postgres (durable log store) |
| Projection / read-model engine | Build + serve materialized read-models (execution/command/runtime state) off the log | The Postgres materializer + projected state tables |
| KV / state engine | Platform key/value + coherence state (chain heads, exec descriptors) | NATS KV |
| Object / blob engine | Platform artifacts, result-tier payloads, Arrow IPC blobs | External object store (GCS/S3) for platform tiers |
| Vector engine | Platform RAG / retrieval (system docs, catalog embeddings) | Qdrant |
These are potentially separate core engines per workload type behind one EHDB catalog/URN namespace, not one storage format bent to every job. The engine set is what makes "one fabric" viable without collapsing distinct access patterns onto a single index.
EHDB serves NoETL platform functionality only. That means: the event log, its projections, platform KV/state, the catalog, the system-WASM store, and platform artifacts/vectors. Nothing else.
Business data is NOT stored in EHDB. A tenant's domain data lives in
real business systems — PostgreSQL, Snowflake, Cassandra, Kafka,
Elasticsearch, ClickHouse, object stores, whatever the business runs —
and NoETL reaches them the normal way: a playbook step using a
plugin/connector (tool: postgres, tool: snowflake, tool: http,
…) under that playbook's policy block, authenticated through the
keychain. Those are external subsystems to NoETL; EHDB has nothing to do
with them.
So when this RFC says "EHDB replaces PostgreSQL / Qdrant / NATS-JetStream / object store," it means NoETL's own internal platform uses of those engines — the event log's transport and store, the platform's KV, the platform's vector index, the platform's artifact store. It never means the business-facing connector targets. A playbook that queries a tenant's Snowflake warehouse is unaffected by EHDB, forever.
This boundary is the load-bearing distinction of the whole program. Keep it prominent in every downstream design.
Decided: the NoETL server↔EHDB relationship is loosely coupled. The
server/API/gateway tier stays control-plane-only and stateless (the
stateless-server-edge shape from the off-server CQRS drive,
noetl/ai-meta#115). EHDB
is bounded data-plane compute embedded in-process in the
worker/system roles (worker-rust src/ehdb, since the Phase B–D
re-home). The integration seam between the two tiers is **the event log
- the catalog/URN namespace**, not a wired per-instance binding. The "which cluster talks to which EHDB instance" relationship is a logical mapping resolved through URN resolution, not a physical per-instance wire.
- Server API is NOT also an EHDB API. The server does not grow a data-plane EHDB surface clients call to read/write EHDB-backed data.
- No 1:1 server↔EHDB instance pinning. Binding a server replica to an EHDB node reintroduces state + affinity at the edge, couples the deploy/scale/failure domains, and caps server scale at storage-node count — the opposite of the off-server drive.
- Stateless, horizontally-scalable edge — server scales on request load independent of storage placement.
- Independent failure/deploy/scale domains — server and EHDB roll independently.
- The seam is already durable — the event log is the append-only source of truth; the catalog/URN namespace is the logical resolution surface; placement can change without redeploying the edge.
Expose a thin EHDB control surface (control-plane, not a data plane): fabric health/readiness, publish/bind/resolve of system-WASM library URNs, catalog/URN resolution. For data-plane retrieval (RAG search, domain-record read/append, stream drain, projection reads) the server routes to the worker/system pool over the drive. It MUST NOT embed an EHDB data plane in its own process.
For an ultra-low-latency control-plane read or a single-node deployment, prefer a control-plane read cache on the server (a read-through cache of catalog / system-WASM resolution / readiness) over making the server the EHDB data plane. It removes the hop while keeping the boundary — the server is still not the authoritative data-plane reader/writer. Explicit exception, not the default.
Note the event-log engine (Decision 2) does not breach this: the server does not read/write the log by reaching into an EHDB node — it appends through the same producer path (below), and the log's storage engine being EHDB is transparent to the edge.
Decided end-state: the NoETL ecosystem becomes self-sufficient with no external infrastructure dependency except the Kubernetes runtime. EHDB replaces NoETL's internal use of PostgreSQL, Qdrant, NATS/JetStream, and the object store across the platform tiers (event log, projections, KV/state, object/blob, vector). Deployed on Kubernetes, a NoETL cluster needs nothing else running to serve the platform.
Each platform tier is backed through a driver interface with per-tier backend selection. EHDB is the default; operators can point any tier back at the incumbent engine per deployment:
| Platform tier | Default (end-state) | Must remain selectable |
|---|---|---|
| Event-log driver | EHDB event-log engine | NATS JetStream + Postgres |
| Projection driver | EHDB projection engine | Postgres materializer |
| KV / state driver | EHDB KV engine | NATS KV |
| Object / blob driver | EHDB object engine | External object store (GCS/S3), Postgres |
| Vector driver | EHDB vector engine | Qdrant |
EHDB-as-default is a consolidation win (one fabric instead of four external dependencies), not a requirement to run NoETL. A deployment can keep JetStream+Postgres for the log and EHDB for vectors, or any per-tier mix. This keeps the business-connector boundary above crisp: the driver interface governs platform tiers only.
The earlier "EHDB never writes the event log" rule is not deleted — it is scoped correctly and still holds, alongside the new event-log engine:
-
Event authorship is unchanged. The event log stays the append-only source of truth. Who may append an application event, and what gets appended, is gated exactly as today: the gateway/server are the gatekeeper of what enters the log through the append-only producer path. Non-log data-plane roles — the worker's bounded data-plane step, the RAG helper, the system-WASM store — still must not fabricate application events. That was always the point of the old rule, and it stands.
-
The storage/ordering/projection engine underneath is now EHDB. What changes is the durable engine that persists, orders, and serves the log and builds its projections — that becomes EHDB's event-log + projection core engines, replacing JetStream + the Postgres materializer. This is a backend change beneath the producer path, not a new event author.
So the two statements are coherent:
- Authorship: gateway/server gate what gets appended; data-plane roles never fabricate events. (unchanged)
- Engine: EHDB is the authoritative persistence + ordering + serving layer for the log and its read-models, invoked through the same append-only producer path. (new)
EHDB writing the log as its storage engine is not the data plane authoring events — it is the disk-and-index under an append that the producer path already authorized.
| Component | Decision 1 (coupling) | Decision 2 (substrate) |
|---|---|---|
| server / API | Control-plane only, stateless. MAY expose a thin EHDB control surface + route data-plane retrieval to the pool. MUST NOT embed a data plane or pin 1:1 to an EHDB instance. Exception: control-plane read cache for an ultra-low-latency / single-node read path. | Appends to the log through the append-only producer path; the log/projection engine underneath is EHDB (or the selected driver). Same gatekeeper role over what is appended; storage engine is transparent. Selects per-tier platform driver via config. |
| worker / playbook / system | Owns EHDB data-plane compute in-process (worker-rust src/ehdb). Bounded, stateless, disabled-by-default, control-plane-guarded. |
Data-plane roles never fabricate application events, on any backend. Ingest writes only the private platform fabric. Business data stays in external systems via connectors — never EHDB. |
| gateway | Gatekeeper only; denied all EHDB data-plane capabilities. | No direct substrate access; gatekeeps what enters the log; unaffected by the driver choice. |
This RFC restructures the roadmap's post-integration phases into the explicit completion trajectory (see Roadmap for detail):
- Phase 6 — Event-Log Core Engine (bottleneck fix). EHDB becomes the durable event-log engine, replacing JetStream+Postgres for the log.
- Phase 7 — Projection / Read-Model Engine. EHDB builds + serves the materialized read-models off the log, retiring the Postgres materializer.
- Phase 8 — KV / State + Object/Blob + Vector Engines. EHDB takes over NoETL's internal NATS-KV, object store, and Qdrant platform tiers. Design + first (KV/state) engine slice landed: Design: KV/State + Object/Blob + Vector Engines (Phase 8).
- Phase 9 — External-Dependency Retirement (Kubernetes-only). The NoETL platform runs self-sufficient on k8s with no external infra dependency for platform functionality.
- Phase 10 — Tunable-Backend Config Surface. The per-tier driver interface making EHDB the default while every tier stays selectable back to the incumbent engine.
- Architecture — execution-model fit, embedded roles, Rust-first worker integration.
- Roadmap — Phase 5 (integration, complete) → Phases 6–10 (completion trajectory above).
- noetl/ehdb#241 — EHDB completion program (decided, implementing).
- noetl/ehdb#234 — EHDB↔NoETL integration umbrella.
- noetl/ai-meta#166 / noetl/ai-meta#104 / noetl/ai-meta#115 — the off-server drive / state-builder / log-bottleneck pressure this program resolves.
- Home
- Architecture
- Architecture — the four engines
- Architecture — resilient KV core
- Consistency Invariants (per tier)
- Roadmap
- Sessions Log
- Claude Handoff
- RFC: Completion Program
- RFC: External EHDB Driver
- L1 Command-Bus Cutover (T4/T5 — prepared, human-gated)
- Prod Cutover — Event-Log Tier (Phase 9, Tier 1)
- Runbook: Async Event-Log Mirror
- Durable Event-Log — Prod Durability Sign-off (§C, slice 6)