Skip to content

Umbrella Event Envelope

Kadyapam edited this page Jun 2, 2026 · 2 revisions

Umbrella — Event Envelope Reconciliation (EE-1..EE-4)

TaskList: #51 in the agent's running task list · Started: 2026-05-29 · Last update: 2026-06-01 · Status: In flight (EE-1 / EE-2 / EE-3 landed; EE-4 not started)

Goal

Reconcile the four divergent shapes of the event envelope that the system carries today so the Rust worker, the Python server, the projector, and the CLI all serialise/deserialise the same bytes for the same conceptual event.

The four shapes

Surfaced via the noetl-server wiki's event envelope page:

Component Type Owns the wire format?
Python server EventEmitRequest Today's source of truth
Rust server EventRequest Mirror of Python (intended)
Rust worker WorkerEvent Drifted — uses an older shape
noetl-executor ExecutorEvent Drifted — uses CLI-local shape

Result: serialising in worker, deserialising in server, requires careful field mapping that's currently inline + fragile.

Phases

EE-1 — snowflake event_id ✅ (landed 2026-05-30)

All four shapes carry a bigint event_id generated by the application via snowflake (per observability.md Principle 3). Server-side gen_snowflake() Postgres function stays as a fallback.

EE-2 — meta.attempts ✅ (landed 2026-05-30)

Every event envelope carries meta.attempts: int (retry count). Projectors group by attempts without reaching back into worker logs.

EE-3 — status taxonomy alignment ✅ (landed 2026-05-31)

call.done status fields normalised across all four shapes: COMPLETED / FAILED / ERROR / CANCELLED. Projector consumers group by the single column without per-source translation.

Shipped in noetl-server v2.0.1 + worker v5.8.0.

EE-4 — type unification ⏳ (not started, blocked on #45 step 1)

Drop WorkerEvent and ExecutorEvent; replace with a shared noetl-events crate that both worker and CLI's local-mode runner consume. Server's Rust port (per Umbrella: Python Services to Rust) will use the same crate, removing the last divergence.

Will likely land as part of the shared library work in #45 step 1 (publisher crate adds the shared library skeleton).

Related

Next concrete steps

EE-4: create noetl-events crate in the same workspace as noetl-tools / noetl-executor. Both worker + executor depend on it. Older types become re-exports during migration.

NoETL Dashboard

Active Umbrellas

Closed Umbrellas

Conventions

Per-repo wikis

Clone this wiki locally