perf: coalesce task completions into batched transactions#59
Merged
Conversation
Spawned tasks now send completions through an unbounded channel instead of individually committing to SQLite. The run loop drains the channel before each dispatch cycle, processing all queued completions in a single BEGIN IMMEDIATE / COMMIT transaction. A leader-election pattern lets spawned tasks opportunistically drain the batch under high concurrency. This amortizes SQLite WAL sync overhead when many tasks complete in a burst.
Contributor
Benchmark ComparisonClick to expand |
deepjoy
pushed a commit
that referenced
this pull request
Mar 19, 2026
## 🤖 New release * `taskmill`: 0.5.1 -> 0.5.2 (✓ API compatible changes) <details><summary><i><b>Changelog</b></i></summary><p> <blockquote> ## [0.5.2](v0.5.1...v0.5.2) - 2026-03-19 ### Other - reduce SQL round-trips and CPU overhead in scheduler hot paths ([#60](#60)) - coalesce task completions into batched transactions ([#59](#59)) - reduce SQL round-trips in scheduler hot paths ([#57](#57)) </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CompletionMsg) so spawned tasks send completions to an unbounded MPSC channel instead of individually committing to SQLitedrain_completions, processing all queued completions in a singleBEGIN IMMEDIATE/COMMITtransaction through the newTaskStore::complete_batch_with_resolvemethodtry_lockon the shared receiver) to opportunistically drain and process the batch inline, reducing latency under high concurrencydrain_completionscall during shutdown ensures no completions are lost before the store closes