Improve type-safety between event payloads and handler maps (Standard Schema / `@standard-schema/spec`)

[dillonstreator]

Today, event type literals (TxOBEvent<T>["type"]) are tracked separately from event.data, which is always Record<string, unknown>. Producers (e.g. createEvent, SQL INSERT into JSONB) and consumers (handlers) therefore cannot share a single source of truth for payload shape at compile time, and runtime validation is left entirely to user code without library-level hooks.

This issue proposes exploring Standard Schema (StandardSchemaV1, also published as @standard-schema/spec) so txob can accept Zod / ArkType / Valibot / … (any spec-compliant validator) as an optional event registry entry and tighten typings end-to-end without depending on a specific schema library.

Current behavior

From the public types (see src/processor.ts):

• TxOBEvent<T> uses data: Record<string, unknown> regardless of T.
• TxOBEventHandler is typed so handlers see TxOBEvent<TxOBEventType> but not a distinct data shape per event type key.
• TxOBEventHandlerMap<EventType> is Record<EventType, Record<string, TxOBEventHandler>> — keys line up with event names, not with payload shapes.

The Postgres example illustrates the gap: EventType is a string union (examples/pg/server.ts), while data is an ad hoc object at insert time; handlers (examples/pg/processor.ts) only see generic event with loose data.

Pain for library consumers

1. Casting and assertions in handlers
   To get a usable payload type, apps typically do one or more of:

   • event.data as MyPayload / as unknown as …
   • Narrowing helpers (if (!isFoo(event.data)) return) without txob knowing the predicate
   • Duplicating types: one type for INSERT, another inferred from Zod — drift-prone

2. No linkage between “publish” and “handle”
   Nothing in txob’s API forces createEvent({ type: "X", data }) to match the payload expected by handlerMap.X. Mistakes surface at runtime (or as silent bugs), not when wiring the map.

3. Validation is duplicated and inconsistent
   Teams often validate in the handler with Zod (etc.), throw ErrorUnprocessableEventHandler on bad rows, and re-implement the same schema at the producer for defense-in-depth — without a shared registry type.

4. Tests work around the union
   Tests use patterns like TxOBEvent<keyof typeof handlerMap> (src/processor.test.ts) which improves event type narrowing but still leaves data untyped.

Why Standard Schema

Standard Schema is a small, dependency-neutral interface (~standard.validate, issues vs value). Libraries like recent Zod / ArkType / Valibot implement it; ecosystem tools can “integrate once” and stay library-agnostic (spec site).

For txob, a registry shaped roughly like:

// Illustrative only — not a final API
type EventSchemas<T extends string> = {
  [K in T]?: StandardSchemaV1<unknown, SomeOutputFor<K>>;
};

could:

• Infer output types for data per event kind (via schema types / inference helpers).
• Optionally validate on enqueue, on dequeue, or at handler boundary with one code path that calls schema['~standard'].validate(...).
• Map validation failure → existing ErrorUnprocessableEventHandler (or a dedicated error) in a documented way.

Validators that do not implement the spec could remain supported via thin adapters or stay “bring your own” inside handlers (status quo).

Paths forward (discussion items)

Direction	Pros	Cons / notes
A. Typed registry + inferred handler map	Strongest TS story: handler for "OrderPlaced" receives typed data.	Requires meaningful generics on EventProcessor, createProcessorClient, createEvent; breaking or additive API design.
B. Validate-only helper (minimal)	Ship validateEventData(schema, event.data) + types that narrow data after success; no change to core maps.	Less ergonomic; producers still untyped unless paired with codegen or duplicate schemas.
C. Opt-in schema field on processor	Backward compatible: schemas?: Partial<Record<EventType, StandardSchemaV1>>.	Define behavior when schema missing (skip vs reject vs dev-only warning).

Cross-cutting questions:

• Where to validate: producer transaction vs processor before handlers vs each handler (latency, duplicate work, idempotency).
• Versioning: schema evolution vs persisted JSONB rows (old payloads) — see Event data schema changes & best practices below.
• DX: export inference helpers from @standard-schema/spec vs minimal copied types to avoid version churn.
• Docs/examples: show Zod + ArkType side-by-side using the same txob API.

Event data schema changes & best practices

Outbox events are often immutable rows (JSONB data written once, replayed many times). Standard Schema helps at validation and inference boundaries; it does not remove the need for an explicit schema evolution strategy. Library docs and examples should call out the following.

Compatibility

• Prefer backward-compatible payload changes: add optional fields with documented defaults, avoid renaming/removing fields without a migration path, and treat unknown keys according to a clear policy (strip vs preserve vs reject).
• For breaking changes, prefer new event types (e.g. OrderPlacedV2) or a discriminated schemaVersion / payloadVersion inside data so handlers can branch or delegate to different validators.
• Align producer and consumer rollout: consumers should tolerate old payloads until backlog drains, or events must be migrated before tightening validation.

Where validation runs

• On enqueue: catches bad writes early but couples the transaction to the current schema; risky if handlers lag deployment.
• On dequeue / before handlers: single place to normalize or reject; good for “runtime truth” vs DB contents.
• Inside handlers: flexible but easy to duplicate rules; Standard Schema helps only if the same schema instance (or equivalent spec) is reused.

Operational practices

• Log or meter validation failures separately from transient handler errors so ops can distinguish bad payloads from infrastructure issues.
• When tightening schemas, use shadow validation (log failures without failing) before enforcing ErrorUnprocessableEventHandler or hard rejects.
• Document whether replays and manual fixes in the DB are supported and how they interact with strict validation.

txob-specific tie-in

• If txob gains a schema registry, document whether missing schema means “accept untyped Record<string, unknown>” (compat mode) vs reject, and how that interacts with already-stored events when schemas change between deployments.

Success criteria (proposal)

• Document current limitations (this issue + README note if desired).
• Spike: EventProcessor (or a thin wrapper) parameterized by a map EventType → payload type derived from Standard Schema outputs.
• Spike: single validation path using StandardSchemaV1 that works with at least two libraries (e.g. Zod + ArkType).
• Decide backward-compatibility story for handlers without schemas.
• Add an example under examples/ that publishes and handles without as on event.data.
• Document schema evolution and rollout guidance (consumer-first vs producer-first, versioning, shadow validation) in README or a dedicated doc linked from this issue.

References

• Standard Schema specification: https://standardschema.dev/schema
• npm: @standard-schema/spec