Limit number of words streaming at a time - useChat

[veerbal1]

Feature Description

Core OpenAI streaming response stream one word at a time e.g

Hello
!
How
can
I
help
you
?

But useChat from ai package stream like approx 3-4 words at a time. e.g

Hello! How
can I help
you?

Screen.Recording.2024-02-02.at.7.11.12.PM.mov

is there any option I can set word streaming limit at a time?

Use Case

For rendering smooth streaming of incoming response instead of flickering

Additional context

No response

[yuichiromukaiyama]

It seems that the Azure OpenAI API will return it that way, so is there a way to deceive it on the front-end...？

https://github.com/Azure-Samples/azure-search-openai-demo/blob/33bc5058e0e7249e78e3b0822ef3cca09d03101a/app/frontend/src/pages/chat/Chat.tsx#L84

[MaxLeiter]

You'll need to handle this with your own code, as streaming can be performance / model dependent.