Skip to content

Data stream improvement: message specific data stream #5046

Closed
@valstu

Description

@valstu

Feature Description

At the moment AI SDK provides a functionality to stream custom data through data stream parts. Custom data is returned directly by the useChat hook. This makes sense if you have some "global" data you want to stream. I have noticed that quite often you need to stream custom data that is related to certain message or even certain tool call. It would be nice if we could have custom data streams appended to certain message.

Use Cases

I think we should have data part that is actually related to certain message. Similarly like we have annotations. Now you have to do all kinds of shenanigans to map data to certain message through the "global" data array returned by the useChat. Just as an example, I have an function call which makes separate llm call and I want to stream that content to "main chat". Now I'm using custom data stream for that, I'm basically streaming deltas from nested llm call through data stream like this:

{
  toolCallId: '123',
  textDelta: 'something'
}

And then map them inside messages to correct function call part so I can render the stream of nested llm call.

It would be nice if I could just access data through messages directly. Something like

messages.map(msg => msg.data)

But in general this would allow us to stream all kinds of information that is message specific. Like if function call happens and that starts a certain workflow, we could stream status updates to the message or like in my example even stream text response from separate llm call.

Additional context

My initial thought was that maybe we should have also partial function call result stream toolResultDeltaStreamPart as described in this issue #3063. But the partial result updates might be just status updates like in the link and not necessarily actual result of the function call so maybe separate data stream per message or function call would be better.

Some related links:
https://x.com/lgrammel/status/1896196147619737801
#3063
#5044 (review)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions