perf(web): remove client freezes during streaming turns - #133
Merged
Conversation
Profiling a live turn showed ~1s of blocked main thread per 15s, in two independent hot paths. formatTimestamp was 855ms of the ~1050ms of client CPU. Sidebar subscribes to sessionsById, whose identity changes on every session event, so each stream flush re-rendered hundreds of persisted rows and each row built two throwaway Intl.DateTimeFormat instances. Cache the formatters, precompute the rows in a memo keyed on the live-id signature, and memoize SessionRow with id-taking callbacks. Markdown ran rehype-highlight with detect: true, sending every unlabelled fenced block through hljs.highlightAuto (386ms at 16KB, 5298ms at 256KB, repaid on every streamed update). Highlight through the shared capped highlightToHtml instead: explicit languages only, 32KB cap, skipped while streaming. Same cases now cost 4ms and 24ms. Also drops the node="[object Object]" attribute react-markdown overrides were spreading onto every pre/code/a.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the client freezes during streaming turns. Diagnosed by profiling, not by guessing.
Baseline
Chrome DevTools
Profiler(200 µs sampling) against a production build, with a live session emitting large tool output:Cause 1 —
formatTimestampin the sidebar: 855 ms, 80% of all client CPUSidebarsubscribes tosessionsById.sessionsByIdon everysession_event.stream-coalescer.tsflushing ~1/s, the entire sidebar re-rendered once per second.formatTimestamp, which constructed two throwawayIntl.DateTimeFormatinstances (toLocaleTimeString+toLocaleDateString). On a list of a few hundred sessions that is several hundred constructors per flush.Fixed on three levels:
lib/utils.ts: cached module-levelIntl.DateTimeFormatinstances, built lazily.Sidebar.tsx: persisted rows (title/subtitle/meta) are precomputed in auseMemokeyed on a signature of the live session ids, not on thesessionsByIdobject — that identity change was what invalidated the memo every second.Sidebar.tsx:SessionRowis memoized, with callbacks that take the rowidso the parent can pass stableuseCallbackidentities instead of a fresh closure per row.Cause 2 — markdown language auto-detection
rehype-highlightran withdetect: true, so every unlabelled fenced block — the most common shape in agent output — went throughhljs.highlightAuto, which scores every registered grammar. And it was repaid on every streamed update of the message.Cost of rendering one markdown message containing a single fenced block (
renderToString, highlight.js 11,commonbundle):hljs.highlightAutois 10-20×hljs.highlight, and the generated HTML inflates the text ~6× (256 KB of code → 1.5 MB of markup).One policy now, in
highlightToHtml(lib/code.tsx), shared by markdown and the tool cards:MAX_HIGHLIGHT_CHARScap of 32 KB (~35 ms measured),CodeBlockno longer callshljs.highlightAuto, and its cap drops from 256 KB (262 ms of highlighting, 1.5 MB of markup) to the shared 32 KB.MaybeJsonBlockchecks the size cap beforetrimStart()/JSON.parse, so a multi-megabyte log is not copied and re-indented just to end up as plain text.Incidental bug fixed
react-markdown passes its source hast node to component overrides as a
nodeprop; ours spread it onto the element, so React 19 serialized it and leftnode="[object Object]"on every markdown<pre>,<code>and<a>in the DOM. Verified at 0 in the browser.KbViewnow reuses the chat markdown renderer (markdownComponents) instead of keeping its own copy of the pipeline, andrehype-highlightis dropped from the dependencies.Measured result
Same scenario, same minified build, after the change:
Rendering is unchanged: highlighted blocks still render, the KB view still renders, no page errors.
Verification
bun testat the root: 1227 pass, 0 fail.bun run --filter '@omp-deck/web' typecheck: clean.lib/markdown.test.tsx) pinning the highlighting contract: explicit language yes, aliases yes, unlabelled no, unknown language no, size cap, just under the cap, streaming, inline code untouched, and<pre>+ copy button./kb.