Description
Feature Description
At the moment AI SDK provides a functionality to stream custom data through data stream parts. Custom data is returned directly by the useChat
hook. This makes sense if you have some "global" data you want to stream. I have noticed that quite often you need to stream custom data that is related to certain message or even certain tool call. It would be nice if we could have custom data streams appended to certain message.
Use Cases
I think we should have data part that is actually related to certain message. Similarly like we have annotations. Now you have to do all kinds of shenanigans to map data to certain message through the "global" data array returned by the useChat
. Just as an example, I have an function call which makes separate llm call and I want to stream that content to "main chat". Now I'm using custom data stream for that, I'm basically streaming deltas from nested llm call through data stream like this:
{
toolCallId: '123',
textDelta: 'something'
}
And then map them inside messages to correct function call part so I can render the stream of nested llm call.
It would be nice if I could just access data through messages directly. Something like
messages.map(msg => msg.data)
But in general this would allow us to stream all kinds of information that is message specific. Like if function call happens and that starts a certain workflow, we could stream status updates to the message or like in my example even stream text response from separate llm call.
Additional context
My initial thought was that maybe we should have also partial function call result stream toolResultDeltaStreamPart
as described in this issue #3063. But the partial result updates might be just status updates like in the link and not necessarily actual result of the function call so maybe separate data stream per message or function call would be better.
Some related links:
https://x.com/lgrammel/status/1896196147619737801
#3063
#5044 (review)