event envelope

Event envelope

The wire format for POST /api/events — what workers (and CLI in distributed mode) send to ingest events into the event log.

Current shape (noetl-server v2.0.1+, after R-1.2 PR-EE-2)

pub struct EventRequest {
    pub execution_id: String,
    pub step: String,
    #[serde(alias = "name")]
    pub event_type: String,
    #[serde(default, alias = "context")]
    pub payload: serde_json::Value,
    pub meta: Option<serde_json::Value>,
    pub worker_id: Option<String>,
    pub result_kind: String,        // "data" | "ref" | "refs"
    pub result_uri: Option<String>,
    pub event_ids: Option<Vec<i64>>,
    pub actionable: bool,
    pub informative: bool,
    // R-1.2 PR-EE-2 additions:
    pub event_id: Option<String>,       // app-side snowflake per observability.md Principle 3
    pub status: Option<String>,         // STARTED / RUNNING / COMPLETED / FAILED
    pub created_at: Option<DateTime<Utc>>,  // wall-clock at emit time
}

Three layered serde aliases keep pre-EE clients working without changes:

• name → event_type (legacy field name)
• context → payload (executor producers send context; server stores as payload)
• Pre-EE clients that omit event_id / status / created_at fall back to server-side defaults (DB snowflake / name-derivation / Utc::now()).

Mirrors the Python EventEmitRequest shape after EE-4 — both servers accept the same wire format.

Cross-stack reconciliation — progress

Tracked on noetl/ai-meta#30 — Appendix H Rust migration umbrella. Reconciliation aligns four shapes (worker WorkerEvent, Python EventEmitRequest, Rust EventRequest, executor ExecutorEvent) so all accept the same wire format and worker switching is a one-liner.

PR	Repo	Status
EE-1	noetl/cli	✅ merged (#37) — executor 0.3.1 enriches ExecutorEvent with optional event_id / worker_id / meta + payload serde alias
EE-2	noetl/server	✅ merged (#6) — this server's EventRequest rename + new optional fields (noetl-server 2.0.0; pipeline-fix #7 published 2.0.1)
EE-4	noetl/noetl (Python)	✅ merged (#639) — Python EventEmitRequest aliases + worker_id top-level field + EventType: Literal[...] → str (the Literal was already out of sync with the dot-notation event types used throughout production Python code)
EE-3	noetl/worker	✅ merged (#11) — noetl-worker 3.0.0: WorkerEvent replaced with ExecutorEvent re-export from noetl_executor::events; EventEmitter stamps worker_id + created_at at the source per observability.md Principle 4

✅ Series complete + validated end-to-end (kind-noetl, 2026-05-31). All four producers / consumers now emit + accept the same wire format on /api/events.

EE-4 finalisation post-merge

EE-4 (noetl/noetl#639) added EventEmitRequest with validation_alias=AliasChoices(...) declarations but the schema lived in dead code — the actually-mounted /api/events endpoint at core/events.py::handle_event used the legacy EventRequest model with no aliases. Surfaced during the noetl-worker (Rust) kind-validation pass: the EE-3 wire shape (event_type, context, execution_id: i64) failed Pydantic validation at the live endpoint.

noetl/noetl#641 (noetl 4.0.0) closes the gap by adding the same validation_alias declarations to core/models.py::EventRequest:

• name ← event_type
• step ← node_name
• payload ← context
• execution_id accepts JSON integer OR string (mode='before' coercion)

After this fix the EE umbrella's wire-shape promise truly holds — every producer's events land cleanly in noetl.event with the EE-3 fields populated. Kind validation summary on noetl/ai-meta#30.

Cross-stack envelope state after EE-1 + EE-2 + EE-3 + EE-4

Source	event_type field	step / node_name field	context / payload field	worker_id field	event_id field
noetl-executor 0.3.1	event_type	step	context (alias: payload)	optional worker_id	optional i64
noetl-server (Rust) v2.0.1+	event_type (alias: name)	step	payload (alias: context)	optional worker_id	optional String
Python EventEmitRequest (v3.0.0+)	event_type (alias: name)	node_name (alias: step)	context (alias: payload)	optional worker_id	optional String
noetl-worker v3.0.0+	event_type	step	context	top-level worker_id	None (server gen_snowflake() default fires; app-side generation follow-up)

All four shapes accept the same wire format on input AND emit it on output. The dual-direction alignment lets workers / executors / projectors switch implementations without breaking event ingestion.

Why ExecutorEvent's shape was the target

• step + status are first-class fields rather than buried in payload — easier for the projector + dashboard queries.
• created_at is stamped at emit time → avoids server-clock skew when ordering events.
• execution_id: i64 matches the Postgres bigint column type directly — no String⇄i64 conversion in the ingest path (the wire still uses String to avoid JSON-number precision loss for large snowflakes in browser clients).
• Documented in executor-crate-architecture as the design target.

Real finding while landing EE-4

Production Python code uses dot-notation event types (step.exit, call.done, command.completed, command.issued, etc.) extensively in noetl/core/dsl/engine/executor/ and noetl/server/api/core/execution.py. But the schema's EventType = Literal["step_completed", "step_started", ...] (underscored, past-tense) only accepted the underscored variants. The strict Literal was either dead code or its validation never actually fired against real worker traffic.

EE-4 loosened EventType to str; semantic validation now lives at the call site (orchestrator + projector) where the real taxonomy is enforced. The dot-notation values are the actual production taxonomy; the Literal was aspirational and never matched reality.

See also

• Rust noetl-server handlers::events
• Python EventEmitRequest
• noetl-worker WorkerEvent
• noetl-executor ExecutorEvent
• executor-crate-architecture (cli wiki)
• Appendix H umbrella — noetl/ai-meta#30