feat(chat): add burst concurrency strategy (vercel/chat#495)#114
Conversation
Port the new `burst` concurrency strategy from upstream commit b75eedb (0.4.29 sync wave). `burst` is a hybrid of `debounce` and `queue`: - On an idle thread the first message waits `debounce_ms` instead of dispatching immediately. Messages that arrive during the window are queued, and when the window elapses the handler is invoked once with the latest message; the earlier burst messages are exposed via `context.skipped`. - After the handler finishes, any messages that arrived while it was running drain identically to `queue` -- the latest is dispatched with the rest in `context.skipped`. This is distinct from `debounce` (which silently drops earlier messages in the burst with no context) and from `queue` (which dispatches the first message immediately without coalescing). Honors `max_queue_size` + `on_queue_full` (queue semantics) and `debounce_ms` (debounce semantics). Existing strategies are unchanged. Implementation notes: - `ConcurrencyStrategy` literal extended to include `"burst"`. - `Chat.handle_incoming_message` dispatches `burst` through the existing `_handle_queue_or_debounce` path. - New branch in `_handle_queue_or_debounce`: enqueue self with `max_queue_size`, sleep `debounce_ms`, extend lock, then `_drain_queue`. - Lock-busy fall-through reuses the queue path -- `effective_max = max_queue_size` and `on_queue_full` is honored (previously gated by `strategy != "debounce"`, which already lets burst through). Tests: 6 new tests in `TestConcurrencyBurst` (`tests/test_chat_faithful.py`) mirror the 5 upstream tests in `chat.test.ts` plus 2 tests that pin the distinct semantics vs `queue` and vs `debounce`. Refs: #98, #108 Upstream: vercel/chat#495 (b75eedb)
|
Warning Review limit reached
More reviews will be available in 30 minutes and 15 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements the 'burst' concurrency strategy in the Python Chat SDK, matching the TypeScript SDK's behavior. The 'burst' strategy waits for a debounce window before invoking the handler, allowing messages that arrive during this window to coalesce into a single dispatch where earlier messages are stored in the skipped context. The changes update the documentation, types, configuration classes, and core message handling logic, and introduce a comprehensive suite of tests to validate the new strategy's behavior. There are no review comments, so we have no feedback to provide.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| # No handler call yet -- still inside the burst window | ||
| assert received_messages == [] | ||
|
|
||
| await task |
| ) | ||
|
|
||
| release_second_handler.set() | ||
| await first_task |
|
|
||
| assert await state.queue_depth("slack:C123:1234.5678") == 2 | ||
|
|
||
| await first_task |
| create_test_message("burst-vs-queue-3", "Hey @slack-bot c"), | ||
| ) | ||
|
|
||
| await task |
| create_test_message("burst-vs-deb-2", "Hey @slack-bot b"), | ||
| ) | ||
|
|
||
| await task |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebcb877f57
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| {"thread_id": thread_id, "lock_key": lock_key}, | ||
| ) | ||
| return | ||
| await self._drain_queue(lock, adapter, thread_id, lock_key) |
There was a problem hiding this comment.
When an idle burst starts with an @-mention but the latest message in the burst does not repeat the mention, this call drains only the latest message into _dispatch_to_handlers. Mention detection then runs on that latest message alone, so an unsubscribed thread never reaches on_mention and the whole turn is silently dropped (e.g. "Hey @slack-bot" followed within debounce_ms by "what time is it?"). Burst needs to carry the mention/routing signal from skipped messages or otherwise dispatch the collapsed turn as a mention.
Useful? React with 👍 / 👎.
2 participants
Ports the new
burstconcurrency strategy from upstream commit b75eedb (vercel/chat#495). Part of the 0.4.29 sync wave (tracking #98, alpha #108).Summary
burstis a hybrid ofdebounceandqueue:debounce_msbefore dispatching. Messages arriving during the window are queued; when the window elapses, the handler is invoked once with the latest message and the earlier burst messages exposed viacontext.skipped.queue-- the latest dispatched, the rest incontext.skipped.max_queue_size+on_queue_full(queue semantics) anddebounce_ms(debounce semantics).drop/queue/debounce/concurrentbehavior is unchanged.Distinct semantics
queuedebounceburstcontext.skippedFiles changed
src/chat_sdk/types.py-- extendConcurrencyStrategyliteral, update docstringssrc/chat_sdk/chat.py-- addburstbranch in dispatch +_handle_queue_or_debouncetests/test_chat_faithful.py-- newTestConcurrencyBurstclass (6 tests)docs/UPSTREAM_SYNC.md-- update strategy list referenceTest plan
uv run ruff check src/ tests/ scripts/-- cleanuv run ruff format --check src/ tests/ scripts/-- 197 files formatteduv run python scripts/audit_test_quality.py-- no new warningsuv run pytest tests/ --tb=short -q-- 4042 passed, 3 skippedTestConcurrencyBurst)Refs: #98, #108
Upstream: vercel/chat#495 (b75eedb)
https://claude.ai/code/session_01FyMxQn2BEAzmwKS1GZczKj