taquba-workflow-v0.5.0
·
71 commits
to master
since this release
Added
Memo: per-step durable key-value store for memoizing within-step
side effects, backed by object storage. Bound to a specific
(run_id, step_number);get(key)/put(key, value)take only
the user key. Strictly per-step; the durable channel between steps
isStepOutcome::Continue's payload, not memo.MemoStore: the backing storeMemoviews are derived from
(Arc<dyn ObjectStore>+ path prefix). Used internally by the
runtime builder; users construct one directly mainly in tests.Step::memo: every step receives aMemoscoped to its own
(run_id, step_number). Runners use it to cache results of
expensive within-step side effects (LLM calls, paid APIs) so
at-least-once retries don't re-pay for work the prior attempt
already did.WorkflowRuntimeBuilder::memo_prefix: configures the object-store
prefixStep::memoentries live under. Defaults to"workflow-memo";
set a distinct prefix when multiple runtimes share one store.Error::Store(taquba::object_store::Error): surfaced from memo
read/write failures. Classified as transient byis_permanent.WorkflowRuntimeBuilder::memo_retention(Duration): opts the runtime
into writing a terminal marker viaMemoStore::write_terminal_marker
on every terminal state (Succeeded, Failed, Cancelled). Markers
outlive the run record and provide the input a memo-retention sweep
consumes to decide when a run's memo entries are eligible for
deletion. Without this setter no marker is written and memo entries
are retained indefinitely (appropriate for short-lived runs or
external cleanup).- Memo-retention sweeper: when
memo_retentionis set,
WorkflowRuntime::runspawns a background task that periodically
scans terminal markers and, for each marker older than the
configured window, deletes the run's memo entries and then the
marker itself. The first sweep fires on startup so a fresh process
catches markers left behind by an earlier one. The sweeper shuts
down with the caller-supplied shutdown future. WorkflowRuntimenow reads every timestamp it writes
(DurableRunRecord::submitted_at_ms, theContinueAfterrun_at,
and the terminal-marker timestamp) through ataquba::Clock. By
default the runtime shares the clock itsQueuewas opened with
(viaQueue::clock), so passing aMockClocktoOpenOptions
virtualises time for the queue and the workflow runtime together.WorkflowRuntimeBuilder::clock(Arc<dyn Clock>)overrides the
defaulted-from-queue clock when a test or specialised setup needs a
separate time source.
Changed
-
Breaking:
WorkflowRuntime::buildernow takes an additional
requiredobject_store: Arc<dyn ObjectStore>argument between the
queue and the runner. The store backsStep::memoand need not be
the same store the queue was opened with, though sharing one (just
cloning theArc) is the common case. Existing call sites must add
the store argument:// Before: let runtime = WorkflowRuntime::builder(queue, runner, hook).build(); // After: let runtime = WorkflowRuntime::builder(queue, store, runner, hook).build();