feat(chat): add queue-debounce concurrency strategy#495
Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
visyat
requested changes
May 13, 2026
Collaborator
visyat
left a comment
visyat
left a comment
There was a problem hiding this comment.
Looks good aside from the mode naming!!
visyat
approved these changes
May 14, 2026
This was referenced May 29, 2026
patrick-chinchill
added a commit
to Chinchill-AI/chat-sdk-python
that referenced
this pull request
May 29, 2026
Ports the new burst concurrency strategy from upstream commit b75eedb (vercel/chat#495). Part of the 0.4.29 sync wave. burst is a hybrid of debounce and queue: - Idle thread: first message waits debounce_ms before dispatching; messages arriving during the window are queued; when the window elapses, the handler runs once with the latest message and earlier burst messages exposed via context.skipped - Busy thread: messages arriving while a handler is running drain identically to queue Distinct semantics from existing strategies: - queue: handler called per-message (queue drains in order) - debounce: handler called once with latest, earlier messages silently dropped - burst: handler called once with latest, earlier messages in context.skipped Honors max_queue_size + on_queue_full (queue semantics) and debounce_ms (debounce semantics). Existing drop / queue / debounce / concurrent behavior unchanged. Naming follows upstream's emitted strategy literal "burst" (not the PR title "queue-debounce") for state-adapter interop. Pre-merge code review verified: §3 pass-interaction clean, §5 no new task lifecycle hazards, §6 ContextVar boundaries clean, wedge tests pin the right invariants.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
adds an opt-in
burstconcurrency strategy for #414when a thread is idle, the first message waits for
debounceMs, messages that arrive during that window are queued, and the handler runs once with the latest message plus earlier burst messages incontext.skippedafter the handler finishes, messages that arrived while it was running are drained like
queue, so the latest queued message is processed with earlier queued messages incontext.skippedkeeps existing
drop,queue,debounce, andconcurrentbehavior unchangedupdates docs to cover
burst, explain when to choose it overdebounce, and document the relatedMessageContextbehavior