`experimental_continueSteps` causes the last chunk to arrive after the tool call

[rennokki]

Description

I tried a lot of settings, and I got to the conclusion that the issue I have is similar to #5493. I tried to raise assistant-ui/assistant-ui#1901, but it was from the Vercel AI SDK in the end.

Pretty weird, if I have a front-end execution of that particular tool, the streaming gets blocked while the chunk is placed after the tool result gets resolved.

So, I have this backend API, where experimental_continueSteps: true:

return streamText({
  model,
  toolCallStreaming: true, // <-- enabled
  tools: allTools,
  messages: coreMessages,
  abortSignal: req.signal,
  maxSteps: 10,
  experimental_continueSteps: true, // <-- this is causing the issue
}).toDataStreamResponse()

The responses are consistently looking like this. Notice the last chunk at the end of the streaming.

f:{"messageId":"msg-ufeQqIOMAnmDfh6MKjUXrJ2G"}
0:"I'll help you "
0:"fix this nasty bug. Let's start"
0:"by checking the current project "
b:{"toolCallId":"toolu_01Wz9D4DPgVfhZ2JwQFWbJja","toolName":"read_files"}
c:{"toolCallId":"toolu_01Wz9D4DPgVfhZ2JwQFWbJja","argsTextDelta":""}
[...]
9:{"toolCallId":"toolu_01Wz9D4DPgVfhZ2JwQFWbJja","toolName":"read_files","args":{"path":"src/app/"}}
0:"structure." # <--- This nasty thing.

If experimental_continueSteps gets disabled, it turns out to be working as expected:

f:{"messageId":"msg-2gYayiQBcVJ63seM78koxL2b"}
0:"I'll help"
0:" you solve this bug. Let's start"
0:" by reading the existing files to unders"
0:"tand the context."
b:{"toolCallId":"toolu_019dAv2H51d35MynE7DViaG4","toolName":"read_files"}
c:{"toolCallId":"toolu_019dAv2H51d35MynE7DViaG4","argsTextDelta":""}
[...]

[rennokki]

Not sure if relevant, but Copilot says the problem is this:

// stream-text.ts

if (
  continueSteps &&
  chunkBuffer.length > 0 &&
  (nextStepType !== 'continue' || // when the next step is a regular step, publish the buffer
    (stepType === 'continue' && !chunkTextPublished)) // when the next step is a continue step, publish the buffer if no text was published in the step
) {

It should have been:

// stream-text.ts

// Always flush the buffer before step-finish, regardless of
// next step type or tool calls, to ensure all text is published
if (
  continueSteps &&
  chunkBuffer.length > 0
) {

[BrentFarese]

We are encountering this as well at Aline.

[riseandignite]

One more clue, disabling smoothStream but keeping experimental_continueSteps: true solves this. ai@4.3.6

[lgrammel]

continueSteps has been removed in ai sdk 5 as models move to larger and larger outputs and this was an experimental feature that caused bugs and complications