Skip to content

taquba-workflow-v0.6.0

Choose a tag to compare

@micllam micllam released this 15 Jun 12:46
· 7 commits to master since this release
73ca489

Changed

  • Terminal marker filenames lead with an inverted timestamp, and the
    memo-retention sweep lists only expired markers (via the
    object-store list_with_offset contract) instead of every retained
    marker on every tick, so a sweep's listing cost is proportional to
    the expired set. MemoStore::list_expired_terminal_markers is the
    new sweeper building block; list_terminal_markers remains for
    inspection. Markers written by earlier versions are not recognised
    by the sweeper: when upgrading a store that ran with
    memo_retention enabled, clear the <memo_prefix>/terminals/
    prefix out-of-band.
  • Step transitions settle atomically. The next step's enqueue (for
    Continue / ContinueAfter) and the terminal run-record delete
    (for terminal outcomes) now join the current step's acknowledgement
    transaction via Taquba's ack_with, halving the durable commits
    per transition and removing the crash window between enqueuing the
    next step and acking the current one: a step's successor exists if
    and only if the step's settlement committed. The terminal hook now
    fires before the settlement commits rather than after the run-record
    delete; hooks remain at-least-once as before.

Fixed

  • WorkflowRuntime::submit no longer serialises every submission on a
    process-wide lock held across queue I/O. The duplicate-check lock is
    now per run id, so concurrent submissions of distinct runs proceed in
    parallel and share WAL group commits. Previously a batch of
    submissions completed at one run per flush interval regardless of
    submission concurrency (about ten runs per second at SlateDB's
    default 100 ms flush); same-run-id submissions keep their existing
    duplicate and input-mismatch semantics.

Added

  • WorkflowRuntimeBuilder::step_output_replay: opt-in
    content-addressed replay of runner-returned step outcomes, keyed by
    (run_id, step_number, SHA-256(step payload)). When enabled, the
    runtime persists every StepOutcome the runner returns (including
    Fail and Cancel) before applying it; if the same step is delivered
    again after a crash before ack, the stored outcome is replayed without
    invoking the runner again. Step errors are not recorded, so retries
    still invoke the runner. A replayed ContinueAfter reduces its delay
    by the time already elapsed since the outcome was stored, preserving
    the original schedule.
  • Memo::content_get and Memo::content_put derive per-step memo keys
    from a MessagePack serialization of caller-supplied input hashed with
    SHA-256.