Skip to content

perf(web): remove client freezes during streaming turns - #133

Merged
jaesbit merged 1 commit into
develfrom
perf/client-render-hot-paths
Jul 25, 2026
Merged

perf(web): remove client freezes during streaming turns#133
jaesbit merged 1 commit into
develfrom
perf/client-render-hot-paths

Conversation

@jaesbit

@jaesbit jaesbit commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Fixes the client freezes during streaming turns. Diagnosed by profiling, not by guessing.

Baseline

Chrome DevTools Profiler (200 µs sampling) against a production build, with a live session emitting large tool output:

  • ~1050 ms of client CPU per 15 s turn, in 18 long tasks.
  • The jank arrived as a ~55 ms task on every coalescer flush (~1/s).

Cause 1 — formatTimestamp in the sidebar: 855 ms, 80% of all client CPU

  1. Sidebar subscribes to sessionsById.
  2. The store builds a fresh object for sessionsById on every session_event.
  3. With stream-coalescer.ts flushing ~1/s, the entire sidebar re-rendered once per second.
  4. Each persisted row called formatTimestamp, which constructed two throwaway Intl.DateTimeFormat instances (toLocaleTimeString + toLocaleDateString). On a list of a few hundred sessions that is several hundred constructors per flush.

Fixed on three levels:

  • lib/utils.ts: cached module-level Intl.DateTimeFormat instances, built lazily.
  • Sidebar.tsx: persisted rows (title/subtitle/meta) are precomputed in a useMemo keyed on a signature of the live session ids, not on the sessionsById object — that identity change was what invalidated the memo every second.
  • Sidebar.tsx: SessionRow is memoized, with callbacks that take the row id so the parent can pass stable useCallback identities instead of a fresh closure per row.

Cause 2 — markdown language auto-detection

rehype-highlight ran with detect: true, so every unlabelled fenced block — the most common shape in agent output — went through hljs.highlightAuto, which scores every registered grammar. And it was repaid on every streamed update of the message.

Cost of rendering one markdown message containing a single fenced block (renderToString, highlight.js 11, common bundle):

fence before after
16 KB unlabelled 386 ms 4 ms
64 KB unlabelled 1241 ms 7 ms
256 KB unlabelled 5298 ms 24 ms
256 KB labelled 1292 ms 29 ms

hljs.highlightAuto is 10-20× hljs.highlight, and the generated HTML inflates the text ~6× (256 KB of code → 1.5 MB of markup).

One policy now, in highlightToHtml (lib/code.tsx), shared by markdown and the tool cards:

  • only an explicit language is highlighted, never an auto-detected one,
  • MAX_HIGHLIGHT_CHARS cap of 32 KB (~35 ms measured),
  • nothing is highlighted while a message is still streaming,
  • above the cap the text still renders in full, just uncoloured — content is never dropped.

CodeBlock no longer calls hljs.highlightAuto, and its cap drops from 256 KB (262 ms of highlighting, 1.5 MB of markup) to the shared 32 KB. MaybeJsonBlock checks the size cap before trimStart()/JSON.parse, so a multi-megabyte log is not copied and re-indented just to end up as plain text.

Incidental bug fixed

react-markdown passes its source hast node to component overrides as a node prop; ours spread it onto the element, so React 19 serialized it and left node="[object Object]" on every markdown <pre>, <code> and <a> in the DOM. Verified at 0 in the browser.

KbView now reuses the chat markdown renderer (markdownComponents) instead of keeping its own copy of the pipeline, and rehype-highlight is dropped from the dependencies.

Measured result

Same scenario, same minified build, after the change:

before after
client CPU ~1050 ms / 15 s 48 ms / 10.7 s
long tasks 18 0

Rendering is unchanged: highlighted blocks still render, the KB view still renders, no page errors.

Verification

  • bun test at the root: 1227 pass, 0 fail.
  • bun run --filter '@omp-deck/web' typecheck: clean.
  • 9 new tests (lib/markdown.test.tsx) pinning the highlighting contract: explicit language yes, aliases yes, unlabelled no, unknown language no, size cap, just under the cap, streaming, inline code untouched, and <pre> + copy button.
  • Manual e2e against a production build in headless Chromium over CDP: heavy session load, a streaming turn, and /kb.

Profiling a live turn showed ~1s of blocked main thread per 15s, in two
independent hot paths.

formatTimestamp was 855ms of the ~1050ms of client CPU. Sidebar
subscribes to sessionsById, whose identity changes on every session
event, so each stream flush re-rendered hundreds of persisted rows and
each row built two throwaway Intl.DateTimeFormat instances. Cache the
formatters, precompute the rows in a memo keyed on the live-id
signature, and memoize SessionRow with id-taking callbacks.

Markdown ran rehype-highlight with detect: true, sending every
unlabelled fenced block through hljs.highlightAuto (386ms at 16KB,
5298ms at 256KB, repaid on every streamed update). Highlight through
the shared capped highlightToHtml instead: explicit languages only,
32KB cap, skipped while streaming. Same cases now cost 4ms and 24ms.

Also drops the node="[object Object]" attribute react-markdown
overrides were spreading onto every pre/code/a.
@jaesbit jaesbit changed the title perf(web): eliminar los cuelgues del cliente durante el streaming perf(web): remove client freezes during streaming turns Jul 25, 2026
@jaesbit
jaesbit merged commit 32eb61e into devel Jul 25, 2026
@jaesbit
jaesbit deleted the perf/client-render-hot-paths branch July 25, 2026 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant