fix(react): throttle useChat messages snapshot to prevent maximum update depth error - #17893
fix(react): throttle useChat messages snapshot to prevent maximum update depth error#17893takumiz19 wants to merge 1 commit into
Conversation
af41232 to
18a7602
Compare
…ate depth error throttle/experimental_throttle only throttled the subscribe callback, while useSyncExternalStore's getSnapshot read `messages` directly and returned a fresh array on every stream chunk. Any re-render then read that un-throttled snapshot and re-rendered per chunk, which could exceed React's nested update limit during fast streaming on slower devices. Publish a separate throttled snapshot from the React chat state so both the change notification and the getSnapshot value are throttled together. Fixes vercel#6166
18a7602 to
1c68f8b
Compare
Bugfix reviewOutcome: changes-required Fixes issueStatus: partially-addresses The stable published snapshot fixes the reported throttle bypass for a single useChat subscriber, but shared Chat instances can still have throttling disabled or changed by another subscriber. Concerns:
Side effectsRisk: medium Moving throttling from individual callbacks into shared Chat state makes previously independent hook options interfere with one another. Concerns:
PerformanceRisk: medium A second unthrottled subscriber can globally disable snapshot throttling and restore the high-frequency rendering behavior responsible for the reported performance failure. Concerns:
Backwards compatibilityRisk: none The change does not alter persisted message formats, storage, serialization, or existing stored data. Breaking changesRisk: medium Public types and accepted inputs remain compatible, but the externally observable semantics of the public throttle option change for shared Chat instances. Concerns:
ArchitectureRisk: medium Package boundaries remain intact, but subscriber-specific React configuration is incorrectly owned by the shared Chat state. Concerns:
Change scopeStatus: minimal The implementation, focused regression test, and patch changeset are all directly related to the reported bug. SecurityRisk: none No credential handling, network validation, parsing, authorization, or other security-sensitive behavior is changed. TestingStatus: needs-more The new test covers unrelated re-renders for one subscriber, but omits supported shared-Chat and subscription-lifecycle cases exposed by the implementation. Concerns:
VerificationInspected all four changed files, the React Chat and AbstractChat state contracts, public exports, documentation examples, and throttle implementation. The @ai-sdk/react suite passed all 90 tests, targeted type checking and diff validation passed, and focused state verification demonstrated that subscriber registration order currently determines the global snapshot throttle. React's official useSyncExternalStore snapshot contract was also checked. Relevant Documentation |
|
closing in favor of #18525 |
## Background `useChat` currently throttles its messages subscription callback, but its `useSyncExternalStore` snapshot always reads the latest `chat.messages` array. Because streaming replaces that array for every chunk, any unrelated React render can observe a new snapshot before the throttled callback publishes it. In high-frequency streams this bypasses `throttle`, causes per-chunk renders, and can contribute to the "Maximum update depth exceeded" failures reported in #6166. ## Summary - Keep a published messages snapshot per `useChat` hook and advance it from that hook's throttled subscription callback. - Publish the latest message snapshot before `ready` or `error` becomes observable, including normal completion and aborts. - Synchronize updates that occur between render and subscription, and ignore delayed callbacks after a hook unsubscribes or changes chat instances. - Add regression coverage for unrelated renders, terminal status/message coherence, aborts, errors, and delayed callbacks after chat replacement. - Turn the existing Next.js throttle route into a deterministic end-to-end reproduction that streams 500 chunks while forcing unrelated renders, reports pass/fail from observed snapshot identities, and verifies the complete message is visible when status becomes `ready`. - Add a patch changeset for `@ai-sdk/react`. ## Contributor Credit - @brahmveda-arkin reported #6166. - @takumiz19 isolated the snapshot/subscription mismatch and proposed #17893. - @ben-reitz demonstrated the practical value of a conservative UI update cadence in cloudflare/agents#2058. ## End-to-End Verification Ran `/chat/throttle` in `examples/ai-e2e-next` in a real browser. The route streams 500 chunks (1,000 assistant characters) with `throttle: 50` while a zero-delay timer independently re-renders the component. - Before: **FAIL**, 255 distinct message snapshots in 1,475ms (maximum expected: 34), across 946 total React renders. - After: **PASS**, 15 distinct message snapshots in 1,144ms (maximum expected: 27), across 602 total React renders. The patched run rendered all 1,000 assistant characters on the same render where status became `ready`, and had no Next.js error overlay or browser error. ## Checklist - [x] All commits are signed (PRs with unsigned commits cannot be merged) - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Future Work This PR intentionally leaves the current opt-in default unchanged, so it does not protect applications that omit `throttle` from high-frequency unthrottled rendering. For v8, I recommend making a 50ms UI publication cadence the default when `throttle` is omitted, with `throttle: 0` as the explicit unthrottled opt-out. Stream processing, tool handling, and callbacks should remain immediate; only snapshots exposed to React should be paced. The default should also guarantee an immediate leading publication and a terminal flush so the final messages and `ready` status stay coherent. This would cap the normal rendering rate at about 20 updates per second and protect applications that do not know they need to opt in today. The tradeoff is up to 50ms of additional visible text latency and an explicit opt-out for applications that intentionally need per-chunk rendering, which makes the behavior change appropriate for a major release. ## Related Issues Addresses the throttled snapshot bypass discussed in #6166. Reports using the default unthrottled behavior remain outside this PR. Closes #17893. Related to cloudflare/agents#2058.
## Background `useChat` in AI SDK 6 throttles its messages subscription callback, but its `useSyncExternalStore` snapshot always reads the latest `chat.messages` array. Because streaming replaces that array for every chunk, an unrelated React render can observe a new snapshot before the throttled callback publishes it. In high-frequency streams this bypasses `experimental_throttle`, causes per-chunk renders, and can contribute to the "Maximum update depth exceeded" failures reported in #6166. This is a branch-native backport of #18525 to `release-v6.0`. ## Summary - Keep a published messages snapshot per `useChat` hook and advance it from that hook's throttled subscription callback. - Publish the latest message snapshot before `ready` or `error` becomes observable, including normal completion and aborts. - Synchronize updates that occur between render and subscription, and ignore delayed callbacks after a hook unsubscribes or changes chat instances. - Add regression coverage for unrelated renders, terminal status/message coherence, aborts, errors, and delayed callbacks after chat replacement. - Turn the existing Next.js throttle route into a deterministic end-to-end reproduction that streams 500 chunks while forcing unrelated renders, reports pass/fail from observed snapshot identities, and verifies the complete message is visible when status becomes `ready`. - Update the reproduction's stale text stream-part shape to the v6 protocol and avoid a render-counter hydration mismatch. - Add a patch changeset for `@ai-sdk/react`. ## Contributor Credit - @brahmveda-arkin reported #6166. - @takumiz19 isolated the snapshot/subscription mismatch and proposed #17893. - @ben-reitz demonstrated the practical value of a conservative UI update cadence in cloudflare/agents#2058. ## Manual Verification Ran `/chat/throttle` in `examples/ai-e2e-next` in a real browser. The route streamed 500 chunks (1,000 assistant characters) with `experimental_throttle: 50` while a zero-delay timer independently re-rendered the component. The patched v6 run passed with 15 distinct message snapshots in 860ms (maximum expected: 22), across 462 total React renders. All 1,000 assistant characters were visible on the first render where status became `ready`, with no Next.js error overlay. Also ran: - `pnpm --filter @ai-sdk/react test -- use-chat.ui.test.tsx` (60 tests passed) - `pnpm check` - `pnpm type-check:full` ## Checklist - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Future Work This backport intentionally leaves the v6 opt-in default unchanged, so applications must continue to set `experimental_throttle` to benefit from paced React publications. For v8, I recommend making a 50ms UI publication cadence the default when `throttle` is omitted, with `throttle: 0` as the explicit unthrottled opt-out. Stream processing, tool handling, and callbacks should remain immediate; only snapshots exposed to React should be paced. The default should guarantee an immediate leading publication and a terminal flush so final messages and `ready` status stay coherent. This would cap the normal rendering rate at about 20 updates per second and protect applications that do not know they need to opt in today. The tradeoff is up to 50ms of additional visible text latency and an explicit opt-out for applications that intentionally need per-chunk rendering, which makes the behavior change appropriate for a major release. ## Related Issues Backport of #18525. Addresses the throttled snapshot bypass discussed in #6166. Reports using the default unthrottled behavior remain outside this PR. Related to cloudflare/agents#2058.
## Background `useChat` in AI SDK 5 throttles its messages subscription callback, but its `useSyncExternalStore` snapshot always reads the latest `chat.messages` array. Because streaming replaces that array for every chunk, an unrelated React render can observe a new snapshot before the throttled callback publishes it. In high-frequency streams this bypasses `experimental_throttle`, causes per-chunk renders, and can contribute to the "Maximum update depth exceeded" failures reported in #6166. This is a branch-native backport of #18525 to `release-v5.0`. ## Summary - Keep a published messages snapshot per `useChat` hook and advance it from that hook's throttled subscription callback. - Publish the latest message snapshot before `ready` or `error` becomes observable, including normal completion and aborts. - Synchronize updates that occur between render and subscription, and ignore delayed callbacks after a hook unsubscribes or changes chat instances. - Add regression coverage for unrelated renders, terminal status/message coherence, aborts, errors, and delayed callbacks after chat replacement. - Turn the existing Next.js throttle route into a deterministic end-to-end reproduction that streams 500 chunks while forcing unrelated renders, reports pass/fail from observed snapshot identities, and verifies the complete message is visible when status becomes `ready`. - Update the reproduction's stale text stream-part shape to the v5 protocol and avoid a render-counter hydration mismatch. - Add a patch changeset for `@ai-sdk/react`. ## Contributor Credit - @brahmveda-arkin reported #6166. - @takumiz19 isolated the snapshot/subscription mismatch and proposed #17893. - @ben-reitz demonstrated the practical value of a conservative UI update cadence in cloudflare/agents#2058. ## Manual Verification Ran `/use-chat-throttle` in `examples/next-openai` in a real browser. The route streamed 500 chunks (1,000 assistant characters) with `experimental_throttle: 50` while a zero-delay timer independently re-rendered the component. The patched v5 run passed with 15 distinct message snapshots in 1,227ms (maximum expected: 29), across 654 total React renders. All 1,000 assistant characters were visible on the first render where status became `ready`, with no Next.js error overlay. Also ran: - `NODE_PATH=packages/rsc/node_modules pnpm --filter @ai-sdk/react test -- use-chat.ui.test.tsx` (53 tests passed; v5's React Vitest config imports the plugin already pinned by the RSC package but does not declare it itself) - `pnpm --filter @ai-sdk/react type-check` - `pnpm --filter @ai-sdk/react... build` - `pnpm check` - `git diff --check` `pnpm type-check:full` remains red in this checkout because unchanged v5 examples resolve incompatible React type versions. It did not report any changed file; the changed React package type-check and declaration build pass, and the changed Next.js route compiled successfully during browser verification. ## Tasks - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] Formatting issues have been fixed (run `pnpm prettier-fix` in the project root) - [x] I have reviewed this pull request (self-review) ## Future Work This backport intentionally leaves the v5 opt-in default unchanged, so applications must continue to set `experimental_throttle` to benefit from paced React publications. For v8, I recommend making a 50ms UI publication cadence the default when `throttle` is omitted, with `throttle: 0` as the explicit unthrottled opt-out. Stream processing, tool handling, and callbacks should remain immediate; only snapshots exposed to React should be paced. The default should guarantee an immediate leading publication and a terminal flush so final messages and `ready` status stay coherent. This would cap the normal rendering rate at about 20 updates per second and protect applications that do not know they need to opt in today. The tradeoff is up to 50ms of additional visible text latency and an explicit opt-out for applications that intentionally need per-chunk rendering, which makes the behavior change appropriate for a major release. ## Related Issues Backport of #18525. Addresses the throttled snapshot bypass discussed in #6166. Reports using the default unthrottled behavior remain outside this PR. Related to cloudflare/agents#2058.
Background
useChatre-renders pastthrottle/experimental_throttleand can hit React's "Maximum update depth exceeded" during streaming on slower devices (Android, heavy markdown, CPU throttling). Reported in #6166.The root cause is a mismatch between what
experimental_throttlethrottles and what actually drives re-renders.useChatsubscribes to messages viauseSyncExternalStore:The throttle was applied only to the subscribe callback (
~registerMessagesCallback). ButreplaceMessageswaps#messagesfor a new array on every chunk, and that is exactly whatgetSnapshot()returned.useSyncExternalStorere-readsgetSnapshot()on every render and, when the reference differs from the rendered one, forces a synchronous re-render (tearing prevention). That path never goes throughsubscribe, so throttling the subscribe callback did not limit it — during streaming the array reference changes on essentially every commit, so React re-renders per chunk regardless of the throttle. Once renders fall behind chunk arrival it trips the nested-update limit.This matches the reports in the thread: it is machine/load dependent (a threshold, not a hard infinite loop), and raising the throttle / using
chunking: 'line'/ memoizing markdown only reduces how much work each forced render does, rather than capping the re-render count.Summary
Keep the published snapshot separate from the internal message buffer so that
getSnapshot()returns a stable reference between throttle windows:ReactChatStatenow holds#messages(internal buffer, always current — used byAbstractChatfor slicing/finding/replacing) and a separate#messagesSnapshot(the reference React reads).#publishMessagesswaps#messagesSnapshot = #messagesand notifies subscribers together, and that whole publish is throttled. So the notification and the snapshot are throttled in lockstep, andexperimental_throttlebehaves as documented.useChat'sgetSnapshotnow readschatRef.current.messagesSnapshotinstead ofmessages.When throttling is disabled (the default) publishing is synchronous, so behavior is unchanged.
End-to-End Verification
Reproduced with a long streamed response under Chrome DevTools 6x CPU throttling: before this change,
experimental_throttle: 100did not prevent "Maximum update depth exceeded"; after it, streaming stays throttled and no longer trips the limit.The added regression test (
does not bypass throttling when the component re-renders for an unrelated reason) fails onmain(an unrelated re-render mid-window shows the un-throttled buffered content) and passes with this change. The full@ai-sdk/reactsuite (90 tests) passes.Checklist
pnpm changesetin the project root)Related Issues
Fixes #6166