perf(playground): virtualize long chat threads with TanStack Virtual by samuv · Pull Request #2192 · stacklok/toolhive-studio

samuv · 2026-05-05T13:56:24Z

Long Playground threads (~200+ messages) felt sluggish on initial render, every streaming tick stuttered, and TanStack Router's scroll restoration on route switch took a noticeable beat — every row stayed mounted and re-rendered on every status update. This PR adopts @tanstack/react-virtual through a hybrid strategy that virtualizes historical rows while keeping the streaming tail in normal flow, so the existing auto-scroll / ResizeObserver follow-to-bottom behaviour stays untouched. The 770-line chat-message monolith is also split into focused per-part modules so the streaming-tick re-render cost stays scoped to the row whose message ref actually changed.

Summary

Hybrid virtualization in chat-message-list.tsx. Threads with messages.length <= VIRTUALIZE_THRESHOLD (10) render flat as before. Longer threads go through VirtualChatMessageList, a small leaf that calls useVirtualizer with measureElement (dynamic row heights), useFlushSync: false (per TanStack's React 19 guidance), and overscan: 5. The leaf carries 'use no memo' + a line-scoped react-hooks/incompatible-library disable so React Compiler does not bail out on the surrounding tree.
Tail-in-flow. The last TAIL_SIZE (2) rows and the "Thinking..." indicator stay in normal flow below the absolutely-positioned virtual rows. The streaming row's natural height growth still drives scrollHeight, so useAutoScroll's follow-to-bottom keeps working with no special-casing for the virtualized path.
DOM decoupling for useAutoScroll in use-auto-scroll.ts. The ResizeObserver now targets a shared data-chat-inner wrapper that exists in both render paths, instead of firstElementChild. Test helper in use-auto-scroll.test.ts is updated to mirror the new DOM shape.
Dropped the index-based animate-in stagger for virtualized rows to avoid visible ghosting as rows recycle through the window; tail rows still get the staggered entrance.
Component split under components/chat-message/. The 770-line chat-message.tsx becomes a 34-line memo(ChatMessageImpl) dispatcher in chat-message/index.tsx that hands off to UserMessage / AssistantMessage. Per-part renderers (reasoning, tool call, tool output, step start, joined assistant text) and adjacent components (mcp-app-view, attachment-preview, image-modal, token-usage, no-content-message) move into the same folder. Public import path is unchanged — ./chat-message resolves to chat-message/index.tsx, matching the existing card-mcp-server / form-run-from-registry / groups-manager idiom.
New tests: per-part component tests under chat-message/__tests__/ (reasoning, tool-call, tool-output) plus a virtualizer-specific test in chat-message-list.test.tsx that asserts the flat path renders every row for short threads, and the virtualized path mounts only a window of historical rows while always keeping the trailing two tail rows in the DOM.

How to validate

Open a Playground thread with 200+ messages → initial render is fast, scrolling is smooth, and only a windowed subset of historical rows lives in the DOM. Inspect under [data-chat-inner]: the tall positioned wrapper holds ~10–20 absolutely-positioned rows, plus 2 tail rows in normal flow.
Send a new message in that long thread → the streaming row keeps auto-scrolling to the bottom as content grows. Stop button still aborts cleanly.
Trigger a tool call producing a tall tool-output mid-thread → the row settles to its real height after the next ResizeObserver tick, no overlap with the next row.
Switch to another route and back → TanStack Router's scroll restoration lands on the saved offset; no flash of empty rows.
Open a fresh thread (≤10 messages) → the flat render path is used and behaviour is identical to before this PR (animate-in stagger present, same DOM shape).
pnpm run type-check, pnpm run lint, and pnpm run test:nonInteractive (28 chat test files / 415 chat tests) all pass locally.

Out of scope

No change to message persistence, the IPC chat transport, or the streaming-reconnect flow from feat(playground): persist streaming responses and survive route changes #2137.
No change to the threads sidebar, tool-results cards, or token-usage rendering.
The component split is folded in here rather than a separate PR because the virtualizer's per-row re-render cost is what makes the split necessary; reviewing them together makes the trade-off legible.

Copilot

Pull request overview

This PR refactors chat message rendering and introduces hybrid virtualization for long Playground threads so historical rows are windowed while the streaming tail stays in normal flow for existing auto-scroll behavior. It fits into the chat feature by reducing render cost in long conversations and splitting the old monolithic message renderer into focused subcomponents.

Changes:

Added ChatMessageList with TanStack Virtual to window historical chat rows while keeping the last two rows and loading indicator in normal flow.
Split the old chat-message.tsx monolith into focused chat-message subcomponents for assistant/user parts, tool output, MCP app views, attachments, and token usage.
Updated useAutoScroll and tests to target a shared [data-chat-inner] wrapper, and added new component/virtualization tests.

Reviewed changes

Copilot reviewed 19 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
renderer/src/features/chat/hooks/use-auto-scroll.ts	Updates auto-scroll to observe a shared inner wrapper across flat and virtualized layouts.
renderer/src/features/chat/hooks/tests/use-auto-scroll.test.ts	Adjusts hook tests to the new DOM anchor shape.
renderer/src/features/chat/components/chat-message/user-message.tsx	Extracts user-message rendering and attachment display from the old monolith.
renderer/src/features/chat/components/chat-message/tool-output-content.tsx	Extracts structured tool output rendering.
renderer/src/features/chat/components/chat-message/tool-call-component.tsx	Extracts expandable tool call UI and status display.
renderer/src/features/chat/components/chat-message/token-usage.tsx	Moves token usage rendering into the chat-message folder and adds streaming placeholder handling.
renderer/src/features/chat/components/chat-message/step-start-component.tsx	Extracts step-boundary rendering.
renderer/src/features/chat/components/chat-message/reasoning-component.tsx	Extracts collapsible reasoning-part rendering.
renderer/src/features/chat/components/chat-message/no-content-message.tsx	Fixes type import path after the component split.
renderer/src/features/chat/components/chat-message/mcp-app-view.tsx	Moves the MCP app iframe/view host into the chat-message folder.
renderer/src/features/chat/components/chat-message/joined-assistant-text.tsx	Extracts assistant text-part joining and markdown rendering.
renderer/src/features/chat/components/chat-message/index.tsx	Adds the new memoized chat-message dispatcher entry point.
renderer/src/features/chat/components/chat-message/image-modal.tsx	Adds the image preview modal used by attachment previews.
renderer/src/features/chat/components/chat-message/attachment-preview.tsx	Adds per-attachment preview cards and modal launch behavior.
renderer/src/features/chat/components/chat-message/assistant-message.tsx	Extracts assistant-message rendering and tool/MCP UI composition.
renderer/src/features/chat/components/chat-message/tests/tool-output-content.test.tsx	Adds focused tests for structured tool output rendering.
renderer/src/features/chat/components/chat-message/tests/tool-call-component.test.tsx	Adds tests for tool call branches and expand/collapse behavior.
renderer/src/features/chat/components/chat-message/tests/reasoning-component.test.tsx	Adds tests for reasoning disclosure behavior.
renderer/src/features/chat/components/chat-message/tests/mcp-app-view.test.tsx	Adds tests for MCP app loading, CSP injection, toolbar, and callbacks.
renderer/src/features/chat/components/chat-message.tsx	Removes the previous monolithic chat-message implementation.
renderer/src/features/chat/components/chat-message-list.tsx	Introduces the hybrid virtualized/flat message list.
renderer/src/features/chat/components/chat-interface.tsx	Swaps the inline message mapping for the new `ChatMessageList`.
renderer/src/features/chat/components/tests/chat-message-list.test.tsx	Adds tests for flat vs virtualized rendering behavior.
pnpm-lock.yaml	Locks the new TanStack Virtual dependency.
package.json	Adds `@tanstack/react-virtual` to dependencies.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Each `<McpAppView>` mount opens a fresh raw MCP client in the main process to call `fetchUiResource`. With virtualized chat threads (see PR #2192) historical rows unmount and remount whenever they leave/re-enter the viewport, which would trigger a reconnect storm on threads with multiple MCP UI cards. Wraps the IPC call in `useQuery` keyed by [serverName, resourceUri] with `staleTime: Infinity` / `gcTime: Infinity` and `retry: false`, so a remount paints from cache immediately and no IPC fires until the resource URI actually changes. Addresses Copilot review on #2192.

The chat-message module had grown to ~770 lines housing the role dispatcher, every part renderer (reasoning, tool call, tool output, step start, joined assistant text), and several components (mcp-app-view, attachment-preview, image-modal, token-usage, no-content-message) that only render inside chat messages. Each piece moves into a focused sibling under components/chat-message/, with the public entry kept at components/chat-message/index.tsx so existing `import { ChatMessage } from "./chat-message"` callsites resolve unchanged. The folder/index layout matches the existing card-mcp-server, form-run-from-registry, and groups-manager idiom. The dispatcher itself shrinks to a small `memo(ChatMessageImpl)` that hands off to UserMessage / AssistantMessage, so streaming ticks only re-render the row whose message ref actually changed and sibling rows skip via shallow prop equality.

Long Playground threads (200+ messages) felt sluggish on initial render, streaming ticks, and route-restored scrolling because every row stayed mounted and re-rendered on every status update. Adopts TanStack Virtual through a hybrid strategy: historical rows render through `useVirtualizer` + `measureElement` (dynamic heights, `useFlushSync: false` per the React 19 guidance, `overscan: 5`), while the last TAIL_SIZE (2) rows and the "Thinking..." indicator stay in normal flow. The streaming row keeps growing scrollHeight naturally, so the ResizeObserver follow-to-bottom logic in useAutoScroll is preserved unchanged. A VIRTUALIZE_THRESHOLD of 10 short-circuits the virtualizer on small threads, and the leaf VirtualChatMessageList carries `use no memo` so React Compiler does not bail out on the surrounding tree. ChatInterface delegates list rendering to the new ChatMessageList, and useAutoScroll finds either render path through a shared `data-chat-inner` wrapper instead of `firstElementChild`.

Each `<McpAppView>` mount opens a fresh raw MCP client in the main process to call `fetchUiResource`. With virtualized chat threads (see PR #2192) historical rows unmount and remount whenever they leave/re-enter the viewport, which would trigger a reconnect storm on threads with multiple MCP UI cards. Wraps the IPC call in `useQuery` keyed by [serverName, resourceUri] with `staleTime: Infinity` / `gcTime: Infinity` and `retry: false`, so a remount paints from cache immediately and no IPC fires until the resource URI actually changes. Addresses Copilot review on #2192.

`<McpAppView>` hosts an iframe whose DOM, bridge connection, and any in-progress user input do not survive an unmount/remount cycle. With the virtualizer in place historical rows recycle as they leave the window, so threads with MCP UI cards would lose user state on every scroll-back. Extends `useVirtualizer`'s `rangeExtractor` so the indices of MCP UI messages are always merged into the visible range. Pinned rows then flow through the same render loop as everything else with correct offsets, and the cache key on `fetchUiResource` from the previous commit makes the rare cold-mount cheap. Non-MCP rows continue to recycle, preserving the perf win. Addresses Copilot review on #2192.

…tualization `ReasoningComponent` and `ToolCallComponent` kept their expand/collapse state in local `useState`, so when virtualized rows recycled in long threads the user would lose the expanded reasoning pane or tool-result view as soon as they scrolled away and back. Lifts the four boolean flags (one for reasoning, three independent slots for the tool-call card) into a small module-level store backed by `useSyncExternalStore`. Each subscriber pulls a key-scoped snapshot, so a toggle on one row never re-renders the others. The components accept an optional `disclosureKey` and fall back to `useId()` when absent, so unrelated callsites and existing tests are unaffected. `<AssistantMessage>` threads `${message.id}:${partIndex}` as the key, which is stable across remounts of the same row. Addresses the remaining Copilot review thread on #2192.

samuv self-assigned this May 5, 2026

Copilot AI review requested due to automatic review settings May 5, 2026 13:56

github-actions Bot added the size/XL label May 5, 2026

Copilot started reviewing on behalf of samuv May 5, 2026 13:57 View session

Copilot AI reviewed May 5, 2026

View reviewed changes

Comment thread renderer/src/features/chat/components/chat-message-list.tsx

Comment thread renderer/src/features/chat/components/chat-message-list.tsx

samuv force-pushed the virtual-thread branch from 25cdcad to 9934703 Compare May 5, 2026 14:24

samuv added 5 commits May 5, 2026 17:15

samuv force-pushed the virtual-thread branch from 9934703 to 59f8a76 Compare May 5, 2026 15:56

peppescg approved these changes May 6, 2026

View reviewed changes

samuv merged commit a3a7af6 into main May 6, 2026
18 checks passed

samuv deleted the virtual-thread branch May 6, 2026 10:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(playground): virtualize long chat threads with TanStack Virtual#2192

perf(playground): virtualize long chat threads with TanStack Virtual#2192
samuv merged 5 commits intomainfrom
virtual-thread

samuv commented May 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

samuv commented May 5, 2026

Summary

How to validate

Out of scope

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants