Skip to content

Conversation

@carlbrugger
Copy link
Contributor

@carlbrugger carlbrugger commented Apr 1, 2025

The word-based regex pattern /\s*\S+\s+/m requires trailing whitespace after each word. The last text-delta won't have a trailing space causing the last word in the last text-delta to not be returned until after any tool calls the model may make. The proposed solution is to match words even if they don't have trailing whitespace.

Resolves #5493

The word-based regex pattern (/\s*\S+\s+/m) requires trailing whitespace after each word. The last text-delta won't have a trailing space causing the last word in the last text-delta to not be returned until after any tool calls the model may make.
@lgrammel
Copy link
Collaborator

lgrammel commented Apr 2, 2025

I have doubts re the replacement regexp. Won't it match way to much, causing half-finished words to be streamed out?

Instead I'll look into flushing the buffer when there is a tool call, which should cover all cases.

@carlbrugger
Copy link
Contributor Author

@lgrammel I've changed the implementation to flush the buffer when the chunk type changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

smoothStream() doesn't properly recognized words

2 participants