State tracking gap during client-defined tool round-trips

[alexanderjacobsen]

Context

This is specifically about client-defined tools — tools registered via the tools option on useAgentChat, with execute functions that run in the browser. This path is documented as the intended way for SDKs and platforms that need to register tools dynamically at runtime, where the server can't know the tool set at deploy time.

During a single user turn that involves one or more client tool calls, the exposed state (status, isStreaming, isServerStreaming) doesn't coherently represent what's actually happening. Two distinct issues, both rooted in the same area.

Issue 1: State gap during client-tool execution

The sequence today:

AI emits tool-call → stream ends                                → status: 'ready', isStreaming: false
SDK invokes onToolCall(...)                                     → status: 'ready', isStreaming: false
  [consumer code awaits tool.execute(), can take seconds]       → status: 'ready', isStreaming: false
addToolOutput(result)                                           → deferred stream created, status: 'submitted', isStreaming: true
Server pushes continuation                                      → isServerStreaming: true

From the consumer's perspective, the entire turn should look like one continuous "something is happening" signal. Instead, there's a real window — the duration of the client-side tool.execute — where no state field reflects any activity. For fast tools this is imperceptible; for anything doing a fetch/API call, it's noticeable.

The root cause is that the deferred stream (which exists specifically to cover this gap) is only created on addToolOutput, not on onToolCall invocation. Since the SDK is the one calling onToolCall, it could easily bracket that invocation with the deferred stream creation. Proposed change:

// Conceptually, inside the SDK's tool dispatch
if (isClientTool(toolCall)) {
  this._createToolContinuationStream();  // moved earlier
  try {
    await options.onToolCall({ toolCall, addToolOutput });
  } catch { /* existing error handling */ }
}

Issue 2: status conflates user-initiated and server-pushed submissions

The new separation between status (user-initiated request lifecycle) and isServerStreaming (server-pushed activity) is the right abstraction. But tool-call continuations currently flip status to 'submitted' via the deferred stream, which breaks the separation — 'submitted' can now mean either:

1. "User just sent a new message, awaiting first token" (a new turn)
2. "Client tool result was just returned, awaiting server continuation" (mid-turn)

Consumers who want to differentiate these (e.g. to show a typing indicator only for #1) have to inspect message history to tell them apart.

A tool-call continuation is inherently server-pushed — the user didn't submit anything. It should bump isServerStreaming, not status. Proposed change:

addToolOutput → isServerStreaming: true → continuation arrives → isServerStreaming stays true → done

With status left untouched, status === 'submitted' becomes unambiguous: it means exactly what it reads, "user submission awaiting response."

Combined proposed state model

Phase	status	isServerStreaming	isStreaming
User sent, waiting for first token	submitted	false	false
First tokens arriving	streaming	false	true
Tool call emitted, onToolCall running	ready	true (from fix 1)	true
addToolOutput fired, awaiting continuation	ready	true	true
Continuation streaming	ready	true (from fix 2)	true
Turn complete	ready	false	false

With this model, consumer code reduces to:

const isLoading = isStreaming || status === 'submitted';
const showTypingIndicator = status === 'submitted';

No counters, no message-history inspection, no refs tracking user-initiated sends. The two state fields cleanly compose and mean exactly what their names suggest.

Why this matters for client-defined tools specifically

Server-side tools with execute run inline within a single streamText call — they never cause the stream to end, so none of these state transitions happen. The entire problem is specific to the client-tool round-trip pattern, which is why it's easy to miss when the primary focus is server-defined tools.

For SDKs and platforms built on top of @cloudflare/ai-chat where tools are client-defined by design, these gaps show up on every tool call. Happy to contribute a PR if the proposed model sounds right — wanted to get alignment on the shape first.

[threepointone]

landed and released #1366 and #1368 that should fix this for you, give it a go with the latest releases?