Skip to content

perf: cache plain text stream checks#545

Closed
sam-saffron-jarvis wants to merge 1 commit intoSamSaffron:mainfrom
sam-saffron-jarvis:feat/webui-perf-pass
Closed

perf: cache plain text stream checks#545
sam-saffron-jarvis wants to merge 1 commit intoSamSaffron:mainfrom
sam-saffron-jarvis:feat/webui-perf-pass

Conversation

@sam-saffron-jarvis
Copy link
Copy Markdown
Contributor

What

  • Adds an incremental plain-text eligibility check for assistant stream tails.
  • Reuses the previous plain-text scan result while streamed content grows by ordinary prose characters.
  • Falls back to the full markdown/math scanner when punctuation or newlines could introduce markdown syntax.

Why

During long plain-text responses, the web UI was rescanning the entire streamed assistant body every render just to decide whether it could keep using the cheap text-node append path. That is unnecessary for the common case where deltas are just words/spaces.

This keeps the conservative markdown behavior, but avoids repeated full scans for the boring case. Boring, in UI perf, is where the money is.

Tests

  • node internal/serveui/static/markdown_streaming_test.js
  • go test ./internal/serveui ./cmd
  • go build ./... && go test ./...

@SamSaffron
Copy link
Copy Markdown
Owner

Consolidated this idea into a single local changeset on main with the other selected PR ideas.

@SamSaffron SamSaffron closed this May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants