Skip to content

feat!: support passing state from execute() to finalize() via typed Memo#69

Merged
deepjoy merged 2 commits into
mainfrom
execute-memo
Mar 22, 2026
Merged

feat!: support passing state from execute() to finalize() via typed Memo#69
deepjoy merged 2 commits into
mainfrom
execute-memo

Conversation

@deepjoy
Copy link
Copy Markdown
Owner

@deepjoy deepjoy commented Mar 22, 2026

Summary

  • Add a default type parameter Memo = () to TypedExecutor<T> so executors can return typed state from execute() that is persisted to SQLite and delivered to finalize() after children complete
  • New Domain::task_memo() and Domain::task_with_memo() registration methods for memo-producing executors; existing Domain::task() is unchanged
  • Add memo BLOB column to tasks and task_history tables (migration 009); TypeId::of::<()>() check avoids DB writes for non-memo tasks
  • Update all doc examples (lib.rs, quick-start.md, migrating-to-0.5.md) to include the new _memo: () parameter in finalize() signatures

Closes #64

Details

The memo is serialized via serde_json at the set_waiting transition (the single correct write point after execute returns and children are detected). On finalize dispatch, the memo bytes are deserialized back into the concrete Memo type. When Memo = (), no serialization or DB write occurs — the TypeId guard short-circuits to Ok(None).

Public API changes

Before After
TypedExecutor<T> TypedExecutor<T, Memo = ()>
execute() -> Result<(), TaskError> execute() -> Result<Memo, TaskError>
finalize(payload, ctx) finalize(payload, memo, ctx)
Domain::task() Domain::task() (unchanged) + Domain::task_memo()
Domain::task_with() Domain::task_with() (unchanged) + Domain::task_with_memo()

Migration

Existing executors with Memo = () only need to add _memo: () to their finalize() override (if any). Executors that don't override finalize() require zero changes.

BREAKING CHANGE

TypedExecutor::finalize() signature adds a Memo parameter between payload and ctx.

deepjoy added 2 commits March 21, 2026 16:22
Add a default type parameter `Memo = ()` to `TypedExecutor<T>` that lets
executors return typed state from `execute()` which is persisted to SQLite
and delivered to `finalize()` after children complete.

- Add `memo BLOB` column to `tasks` and `task_history` (migration 009)
- `TypedExecutor::execute()` returns `Result<Memo, TaskError>`
- `TypedExecutor::finalize()` receives the `Memo` as a parameter
- Memo is serialized at `set_waiting` (the single correct write point)
- `TypeId::of::<()>()` check avoids DB writes for non-memo tasks
- `Domain::task::<T>()` unchanged; new `Domain::task_memo()` for memo executors
- Existing executors with `Memo = ()` require zero code changes
- Executors overriding `finalize()` add `_memo: ()` parameter

BREAKING CHANGE: `TypedExecutor::finalize()` signature adds a `Memo`
parameter between `payload` and `ctx`.
@deepjoy deepjoy enabled auto-merge (squash) March 22, 2026 00:44
@deepjoy deepjoy merged commit e91f356 into main Mar 22, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Mar 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Comparison

Click to expand
group                                       current
-----                                       -------
backoff_delay/constant                      1.00     43.6±0.33ns        ? ?/sec
backoff_delay/exponential                   1.00    194.8±1.22ns        ? ?/sec
backoff_delay/exponential_jitter            1.00    409.0±2.25ns        ? ?/sec
backoff_delay/linear                        1.00     80.2±0.32ns        ? ?/sec
batch_submit_1000                           1.00     36.3±3.55ms        ? ?/sec
byte_progress/byte_reporting_500            1.00    270.4±5.79ms        ? ?/sec
byte_progress/noop_500                      1.00    267.4±5.29ms        ? ?/sec
byte_progress_snapshot_100_tasks            1.00     88.4±2.73ms        ? ?/sec
concurrency_scaling/1                       1.00    408.7±7.46ms        ? ?/sec
concurrency_scaling/2                       1.00    343.6±5.46ms        ? ?/sec
concurrency_scaling/4                       1.00    297.2±6.21ms        ? ?/sec
concurrency_scaling/8                       1.00    267.2±4.87ms        ? ?/sec
count_by_tags/100                           1.00    134.9±4.84µs        ? ?/sec
count_by_tags/1000                          1.00    228.7±6.39µs        ? ?/sec
count_by_tags/5000                          1.00    623.2±9.37µs        ? ?/sec
dep_chain_dispatch/10                       1.00     12.4±0.14ms        ? ?/sec
dep_chain_dispatch/25                       1.00     29.7±0.44ms        ? ?/sec
dep_chain_dispatch/50                       1.00     59.1±1.15ms        ? ?/sec
dep_chain_submit/10                         1.00      3.6±0.21ms        ? ?/sec
dep_chain_submit/200                        1.00     82.6±5.67ms        ? ?/sec
dep_chain_submit/50                         1.00     18.9±1.21ms        ? ?/sec
dep_fan_in_dispatch/10                      1.00      8.0±0.11ms        ? ?/sec
dep_fan_in_dispatch/100                     1.00     58.8±1.01ms        ? ?/sec
dep_fan_in_dispatch/50                      1.00     30.5±0.72ms        ? ?/sec
dispatch_and_complete_1000                  1.00    535.1±8.39ms        ? ?/sec
dispatch_group_scaling/1                    1.00    457.2±9.67ms        ? ?/sec
dispatch_group_scaling/10                   1.00    456.8±5.96ms        ? ?/sec
dispatch_group_scaling/100                  1.00    457.8±6.94ms        ? ?/sec
dispatch_group_scaling/50                   1.00    456.8±5.93ms        ? ?/sec
dispatch_no_groups_500                      1.00    265.4±4.48ms        ? ?/sec
dispatch_one_group_500                      1.00    457.3±6.95ms        ? ?/sec
dispatch_permanent_failure_500              1.00    403.3±5.34ms        ? ?/sec
history_by_type/100                         1.00   924.6±18.88µs        ? ?/sec
history_by_type/1000                        1.00   960.8±37.20µs        ? ?/sec
history_by_type/5000                        1.00  1000.1±26.55µs        ? ?/sec
history_query/100                           1.00   567.3±15.13µs        ? ?/sec
history_query/1000                          1.00   575.4±11.13µs        ? ?/sec
history_query/5000                          1.00   589.1±11.20µs        ? ?/sec
history_stats/100                           1.00    154.5±1.78µs        ? ?/sec
history_stats/1000                          1.00    371.1±1.30µs        ? ?/sec
history_stats/5000                          1.00   1332.2±4.35µs        ? ?/sec
mixed_priority_dispatch_500                 1.00    299.0±5.37ms        ? ?/sec
peek_next/100                               1.00    128.6±4.37µs        ? ?/sec
peek_next/1000                              1.00    130.4±4.37µs        ? ?/sec
peek_next/5000                              1.00    130.3±4.87µs        ? ?/sec
query_by_tags/100                           1.00  1264.3±161.99µs        ? ?/sec
query_by_tags/1000                          1.00     10.9±1.54ms        ? ?/sec
query_by_tags/5000                          1.00     57.0±4.71ms        ? ?/sec
retryable_dead_letter/constant              1.00    164.5±2.60ms        ? ?/sec
retryable_dead_letter/exponential           1.00    165.1±1.84ms        ? ?/sec
retryable_dead_letter/exponential_jitter    1.00    165.2±2.39ms        ? ?/sec
retryable_dead_letter/linear                1.00    164.6±2.02ms        ? ?/sec
submit_1000_tasks                           1.00    195.2±6.58ms        ? ?/sec
submit_dedup_hit_1000                       1.00   258.5±11.05ms        ? ?/sec
submit_with_tags/0                          1.00     98.1±4.86ms        ? ?/sec
submit_with_tags/10                         1.00   257.3±15.08ms        ? ?/sec
submit_with_tags/20                         1.00   419.0±22.87ms        ? ?/sec
submit_with_tags/5                          1.00    179.8±9.85ms        ? ?/sec
tag_values/100                              1.00    144.0±5.75µs        ? ?/sec
tag_values/1000                             1.00    205.1±7.19µs        ? ?/sec
tag_values/5000                             1.00    466.7±7.70µs        ? ?/sec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support passing state from execute() to finalize()

1 participant