[Feature] Saga / Process Manager with compensation

[pathosDev]

Size / Priority

• Size: L
• Reference: Lagom Sagas, Vlingo Process, MassTransit Saga State Machine.

Rationale

The actor model handles fail-stop scenarios well (supervision, restart). It does NOT natively handle business-level multi-step workflows with compensating actions:

• "Book hotel → book flight → book car. If car fails, cancel flight + hotel."
• "Reserve inventory → charge card → ship. If charge fails, release reservation."

Today, users either roll their own state-machine inside a PersistentActor (verbose, error-prone) or pull in a separate workflow framework. Sagas / Process Managers are the canonical pattern:

• Each step has a forward action + a compensating action.
• On step failure, compensations run in reverse order.
• State persisted at every step so failures recover.

This is L because it needs durable workflow state, compensation registry, retry/timeout policies, integration with PersistentActor. Not XL — well-bounded.

Reference: what Lagom does

class OrderSaga extends Saga {
  val steps = List(
    Step("reserve-inventory",
      action = sendCmd(InventoryService, ReserveItem(orderId)),
      compensate = sendCmd(InventoryService, ReleaseItem(orderId))),
    Step("charge-card",
      action = sendCmd(PaymentService, Charge(amount)),
      compensate = sendCmd(PaymentService, Refund(amount))),
    Step("ship",
      action = sendCmd(ShippingService, Ship(orderId)),
      compensate = noop),    // shipped — can't compensate
  )
}

On failure of step N: steps 1..N-1 compensated in reverse order.

Design sketch — actor-ts equivalent

// src/saga/Saga.ts (new)

export interface SagaStep<TInput, TOutput> {
  readonly name: string;
  /** Forward action.  Returns the result (passed to next step). */
  forward(input: TInput): Promise<TOutput>;
  /** Compensating action.  Called with the FORWARD's output if forward succeeded. */
  compensate?: (forwardOutput: TOutput) => Promise<void>;
  /** Retry policy for the forward action.  Default: no retry. */
  retry?: { readonly maxAttempts: number; readonly backoffMs: number };
  /** Timeout for the forward action.  Default: 30s. */
  timeoutMs?: number;
}

export abstract class Saga<TContext> extends PersistentActor<SagaCmd, SagaEvent, SagaState<TContext>> {
  abstract steps(): ReadonlyArray<SagaStep<unknown, unknown>>;
  abstract initialContext(): TContext;

  // PersistentActor base handles:
  //   - Persist (StepStarted, StepCompleted, StepFailed, CompensationStarted, CompensationCompleted) events
  //   - Replay rebuilds saga state on recovery → resumes from last incomplete step
  //   - Auto-run compensations on terminal failure
  //   - Emit SagaCompleted / SagaFailed signal on terminal
}

State tracked:

interface SagaState<TContext> {
  readonly status: 'running' | 'compensating' | 'completed' | 'failed';
  readonly completedSteps: ReadonlyArray<{ name: string; output: unknown }>;
  readonly currentStepIndex: number;
  readonly compensationsRun: ReadonlyArray<string>;
  readonly context: TContext;
  readonly failure?: { readonly atStep: string; readonly error: string };
}

Lifecycle:

1. Saga starts; runs steps[0].forward.
2. On success: persist StepCompleted; recurse with steps[1].
3. On failure: persist StepFailed; switch to compensating; run steps[N-1..0].compensate in reverse.
4. On compensation complete: persist SagaFailed; emit signal.
5. On all-steps complete: persist SagaCompleted; emit signal.

Recovery: replay events; resume from currentStepIndex (whether in forward or compensating phase).

Integration with existing actor-ts subsystems

• PersistentActor: base class; saga inherits.
• BackoffSupervisor: optional wrapping if saga itself crashes.
• Inbox ([Feature] Inbox — actor-level dedup for non-actor callers #181): external triggers can be dedup'd.
• Cluster sharding: sharded sagas — one saga per business-key (e.g. orderId).
• Metrics: saga progress gauges, step duration histograms.

Out of scope / non-goals

• Parallel steps (multi-step concurrent forward) — phase 1: sequential only. Phase 2: parallel groups.
• Choreography saga (each service publishes events, others react) — phase 1: orchestration only (single saga actor coordinates).
• Visual workflow designer — code-defined only.
• Distributed transactions — saga is the eventual-consistency alternative to [Feature] Transactional state (ACID 2PC across grains) #171's 2PC.

Open design questions

1. Compensation failure handling: what if a compensation itself fails? Retry policy on compensations? Recommend: retry per saga config; manual intervention on permanent failure (alert metric).
2. Step idempotency: forward and compensate must be idempotent (in case of retry/replay). Document; framework can't enforce.
3. Saga timeout (whole-saga): separate from step-timeout. Recommend: optional sagaTimeoutMs; if exceeded, switch to compensating regardless of current step.
4. External-call abstraction: steps interact with external services (HTTP, other actors). Sketch uses arbitrary async functions. Add explicit actorCall / httpCall helpers for common cases? Recommend: keep generic.

Test plan

1. Order-saga happy path — all 3 steps succeed; SagaCompleted emitted.
2. Step 2 fails — step 1 compensated; SagaFailed emitted.
3. Saga crash mid-execution — restart; resumes from last persisted step.
4. Step retry — step 1 fails 2× with retry config (max 3); succeeds on attempt 3.
5. Step timeout — forward exceeds timeoutMs; treated as failure; compensations run.
6. Compensation failure — compensation fails; retried; eventual manual-alert metric on permanent fail.
7. Idempotent forward replay — saga restart re-runs step that previously committed; idempotency handles.
8. Sharded saga — one per orderId; concurrent sagas isolated.
9. Cross-runtime — Bun, Node, Deno.

Acceptance criteria

• Saga<TContext> base class extending PersistentActor.
• SagaStep<TInput, TOutput> interface.
• Forward + compensate + retry + timeout per step.
• Recovery via event replay.
• Compensation runs in reverse on failure.
• Metrics + signals.
• Documentation: "Saga vs Transaction ([Feature] Transactional state (ACID 2PC across grains) #171)" decision guide; full bank-transfer-with-compensation example.
• Test suite covers all 9 cases.
• CHANGELOG entry under "New: Saga / Process Manager".

[pathosDev]

Folding in #88, which covered the same pattern and is being closed in favour of this issue. Preserving its content here so nothing is lost.

API sketch from #88:

const result = await Saga.run([
  {
    do: async () => bookingActor.tell({ kind: 'reserve', flightId }),
    compensate: async () => bookingActor.tell({ kind: 'release', flightId }),
  },
  {
    do: async () => paymentActor.tell({ kind: 'charge', amount }),
    compensate: async () => paymentActor.tell({ kind: 'refund', amount }),
  },
  {
    do: async () => emailActor.tell({ kind: 'send', template: 'confirmation' }),
    // no compensation — emails can't be unsent, but step 3 is the last one anyway
  },
]);

Scope notes from #88:

• Saga.run(steps, opts?) runs steps in order; on failure it runs compensate for all completed steps in reverse.
• opts.persistenceId? enables crash-safe sagas — each step's status is written to a journal so a restart resumes where it stopped. Built on PersistentActor when set, pure in-memory otherwise.
• Explicitly out of scope there: cross-process saga coordination (the saga lives in one actor; the steps it triggers can be cluster-wide), and multi-hour sagas whose state lives in DistributedData.
• Estimate given there: 4–5 days (against this issue's L sizing, which additionally covers the compensation registry and retry/timeout policies).

Worth noting the sketch uses tell for each step, which cannot detect failure — a real implementation needs ask or an explicit completion signal per step. That is a gap in the sketch, not a decision to carry over.