Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/adapters/anthropic.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ const stream = chat({

### Thinking (Extended Thinking)

Enable extended thinking with a token budget. This allows Claude to show its reasoning process, which is streamed as `thinking` chunks and displayed as `ThinkingPart` in messages:
Enable extended thinking with a token budget. This allows Claude to show its reasoning process, which is streamed as `STEP_STARTED` and `STEP_FINISHED` events and displayed as `ThinkingPart` in messages:

```typescript
providerOptions: {
Expand Down
2 changes: 1 addition & 1 deletion docs/adapters/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ const stream = chat({

### Reasoning

Enable reasoning for models that support it (e.g., GPT-5). This allows the model to show its reasoning process, which is streamed as `thinking` chunks:
Enable reasoning for models that support it (e.g., GPT-5). This allows the model to show its reasoning process, which is streamed as `STEP_STARTED` and `STEP_FINISHED` events:

```typescript
providerOptions: {
Expand Down
75 changes: 55 additions & 20 deletions docs/api/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,33 +247,68 @@ interface ModelMessage {

### `StreamChunk`

TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction) for streaming. All events share a common base structure:

```typescript
type StreamChunk =
| ContentStreamChunk
| ThinkingStreamChunk
| ToolCallStreamChunk
| ToolResultStreamChunk
| DoneStreamChunk
| ErrorStreamChunk;

interface ThinkingStreamChunk {
type: "thinking";
id: string;
model: string;
interface BaseEvent {
type: EventType;
timestamp: number;
delta?: string; // Incremental thinking token
model?: string;
rawEvent?: unknown;
}

type EventType =
| 'RUN_STARTED' // Run lifecycle begins
| 'RUN_FINISHED' // Run completed successfully
| 'RUN_ERROR' // Error occurred
| 'TEXT_MESSAGE_START' // Text message begins
| 'TEXT_MESSAGE_CONTENT' // Text content streaming
| 'TEXT_MESSAGE_END' // Text message completes
| 'TOOL_CALL_START' // Tool invocation begins
| 'TOOL_CALL_ARGS' // Tool arguments streaming
| 'TOOL_CALL_END' // Tool call completes (with result)
| 'STEP_STARTED' // Thinking/reasoning step begins
| 'STEP_FINISHED' // Thinking/reasoning step completes
| 'STATE_SNAPSHOT' // Full state synchronization
| 'STATE_DELTA' // Incremental state update
| 'CUSTOM'; // Custom extensibility events

type StreamChunk =
| RunStartedEvent
| RunFinishedEvent
| RunErrorEvent
| TextMessageStartEvent
| TextMessageContentEvent
| TextMessageEndEvent
| ToolCallStartEvent
| ToolCallArgsEvent
| ToolCallEndEvent
| StepStartedEvent
| StepFinishedEvent
| StateSnapshotEvent
| StateDeltaEvent
| CustomEvent;

// Example: Thinking/reasoning event
interface StepFinishedEvent extends BaseEvent {
type: "STEP_FINISHED";
stepId: string;
delta?: string; // Incremental thinking token
content: string; // Accumulated thinking content
}
```

Stream chunks represent different types of data in the stream:
Stream events represent different types of data in the stream:

- **`RUN_STARTED` / `RUN_FINISHED`** - Run lifecycle events
- **`TEXT_MESSAGE_*`** - Text content being generated
- **`STEP_STARTED` / `STEP_FINISHED`** - Model's reasoning process (thinking)
- **`TOOL_CALL_*`** - Tool invocation and results
- **`RUN_ERROR`** - Stream errors
- **`STATE_*`** - Shared state updates
- **`CUSTOM`** - Custom extensibility events

- **Content chunks** - Text content being generated
- **Thinking chunks** - Model's reasoning process (when supported by the model)
- **Tool call chunks** - When the model calls a tool
- **Tool result chunks** - Results from tool execution
- **Done chunks** - Stream completion
- **Error chunks** - Stream errors
See [AG-UI Event Definitions](../protocol/chunk-definitions) for full details.

### `Tool`

Expand Down
10 changes: 5 additions & 5 deletions docs/guides/agentic-cycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,21 +46,21 @@ sequenceDiagram

Note over LLM: Cycle 1: Call first tool

LLM->>Server: tool_call: get_weather(SF)
LLM->>Server: TOOL_CALL_START/ARGS: get_weather(SF)
Server->>Tools: Execute get_weather
Tools-->>Server: {temp: 65, conditions: "sunny"}
Server->>LLM: tool_result
Server->>LLM: TOOL_CALL_END with result

Note over LLM: Cycle 2: Call second tool

LLM->>Server: tool_call: get_weather(LA)
LLM->>Server: TOOL_CALL_START/ARGS: get_weather(LA)
Server->>Tools: Execute get_weather
Tools-->>Server: {temp: 75, conditions: "clear"}
Server->>LLM: tool_result
Server->>LLM: TOOL_CALL_END with result

Note over LLM: Cycle 3: Generate answer

LLM-->>Server: content: "SF is 65°F..."
LLM-->>Server: TEXT_MESSAGE_CONTENT: "SF is 65°F..."
Server-->>Client: Stream response
Client->>User: Display answer
```
Expand Down
36 changes: 19 additions & 17 deletions docs/guides/client-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,24 @@ sequenceDiagram
participant Browser
participant ClientTool

LLM Service->>Server: tool_call chunk<br/>{name: "updateUI", args: {...}}
LLM Service->>Server: TOOL_CALL_START event<br/>{toolName: "updateUI", toolCallId: "..."}
LLM Service->>Server: TOOL_CALL_ARGS event<br/>{delta: "{...}"}
Server->>Server: Check if tool has<br/>server execute

Note over Server: No execute function<br/>= client tool

Server->>Browser: Forward tool-input-available<br/>chunk via SSE/HTTP
Browser->>Browser: onToolCall callback<br/>triggered
Server->>Browser: Forward CUSTOM event<br/>(tool-input-available) via SSE/HTTP
Browser->>Browser: Client tool handler<br/>triggered
Browser->>ClientTool: execute(args)
ClientTool->>ClientTool: Update UI,<br/>localStorage, etc.
ClientTool-->>Browser: Return result
Browser->>Server: POST tool result
Server->>LLM Service: Add tool_result<br/>to conversation
Server->>LLM Service: Add TOOL_CALL_END with<br/>result to conversation

Note over LLM Service: Model uses result<br/>to continue

LLM Service-->>Server: Stream response
Server-->>Browser: Forward chunks
LLM Service-->>Server: Stream TEXT_MESSAGE_CONTENT events
Server-->>Browser: Forward events
```

## When to Use Client Tools
Expand All @@ -41,15 +42,16 @@ sequenceDiagram

## How It Works

1. **Tool Call from LLM**: LLM decides to call a client tool
2. **Server Detection**: Server sees the tool has no `execute` function
3. **Client Notification**: Server sends a `tool-input-available` chunk to the browser
4. **Client Execution**: Browser's `onToolCall` callback is triggered with:
1. **Tool Call from LLM**: LLM decides to call a client tool via `TOOL_CALL_START` event
2. **Arguments Streaming**: Tool arguments stream via `TOOL_CALL_ARGS` events
3. **Server Detection**: Server sees the tool has no `execute` function
4. **Client Notification**: Server sends a `CUSTOM` event (name: `tool-input-available`) to the browser
5. **Client Execution**: Browser's client tool handler is triggered with:
- `toolName`: Name of the tool to execute
- `input`: Parsed arguments
5. **Result Return**: Client executes the tool and returns the result
6. **Server Update**: Result is sent back to the server and added to the conversation
7. **LLM Continuation**: LLM receives the result and continues the conversation
6. **Result Return**: Client executes the tool and returns the result
7. **Server Update**: Result is sent back as a `TOOL_CALL_END` event with the result
8. **LLM Continuation**: LLM receives the result and continues the conversation

## Defining Client Tools

Expand Down Expand Up @@ -199,12 +201,12 @@ function MessageComponent({ message }: { message: ChatMessages[number] }) {

## Automatic Execution

Client tools are **automatically executed** when the model calls them. No manual `onToolCall` callback needed! The flow is:
Client tools are **automatically executed** when the model calls them. No manual callback needed! The flow is:

1. LLM calls a client tool
2. Server sends `tool-input-available` chunk to browser
1. LLM calls a client tool via `TOOL_CALL_START` and `TOOL_CALL_ARGS` events
2. Server sends `CUSTOM` event (name: `tool-input-available`) to browser
3. Client automatically executes the matching tool implementation
4. Result is sent back to server
4. Result is sent back to server as a `TOOL_CALL_END` event
5. Conversation continues with the result

## Type Safety Benefits
Expand Down
11 changes: 6 additions & 5 deletions docs/guides/server-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,25 @@ sequenceDiagram
participant Tool
participant Database/API

LLM Service->>Server: tool_call chunk<br/>{name: "getUserData", args: {...}}
LLM Service->>Server: TOOL_CALL_START event<br/>{toolName: "getUserData", toolCallId: "..."}
LLM Service->>Server: TOOL_CALL_ARGS event<br/>{delta: "{...}"}
Server->>Server: Parse tool call<br/>arguments
Server->>Tool: execute(parsedArgs)
Tool->>Database/API: Query/Fetch data
Database/API-->>Tool: Return data
Tool-->>Server: Return result
Server->>Server: Create tool_result<br/>message
Server->>LLM Service: Continue chat with<br/>tool_result in history
Server->>Server: Create TOOL_CALL_END<br/>with result
Server->>LLM Service: Continue chat with<br/>tool result in history

Note over LLM Service: Model uses result<br/>to generate response

LLM Service-->>Server: Stream content chunks
LLM Service-->>Server: Stream TEXT_MESSAGE_CONTENT events
Server-->>Server: Stream to client
```

## How It Works

1. **Tool Call Received**: Server receives a `tool_call` chunk from the LLM
1. **Tool Call Received**: Server receives `TOOL_CALL_START` and `TOOL_CALL_ARGS` events from the LLM
2. **Argument Parsing**: The tool arguments (JSON string) are parsed and validated against the input schema
3. **Execution**: The tool's `execute` function is called with the parsed arguments
4. **Result Processing**: The result is:
Expand Down
26 changes: 15 additions & 11 deletions docs/guides/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,30 +64,34 @@ messages.forEach((message) => {
});
```

## Stream Chunks
## Stream Events

Stream chunks contain different types of data:
TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction) for streaming. Stream events contain different types of data:

- **Content chunks** - Text content being generated
- **Thinking chunks** - Model's internal reasoning process (when supported)
- **Tool call chunks** - When the model calls a tool
- **Tool result chunks** - Results from tool execution
- **Done chunks** - Stream completion
- **`RUN_STARTED` / `RUN_FINISHED`** - Run lifecycle events
- **`TEXT_MESSAGE_START` / `TEXT_MESSAGE_CONTENT` / `TEXT_MESSAGE_END`** - Text content streaming
- **`STEP_STARTED` / `STEP_FINISHED`** - Model's internal reasoning process (thinking)
- **`TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END`** - Tool invocation and results
- **`STATE_SNAPSHOT` / `STATE_DELTA`** - Shared state updates
- **`CUSTOM`** - Custom extensibility events

### Thinking Chunks
### Thinking Events

Thinking chunks represent the model's reasoning process. They stream separately from the final response text:
Thinking events (`STEP_STARTED` / `STEP_FINISHED`) represent the model's reasoning process. They stream separately from the final response text:

```typescript
for await (const chunk of stream) {
if (chunk.type === "thinking") {
if (chunk.type === "STEP_STARTED") {
console.log("Thinking started:", chunk.stepId);
}
if (chunk.type === "STEP_FINISHED") {
console.log("Thinking:", chunk.content); // Accumulated thinking content
console.log("Delta:", chunk.delta); // Incremental thinking token
}
}
```

Thinking chunks are automatically converted to `ThinkingPart` in `UIMessage` objects. They are UI-only and excluded from messages sent back to the model.
Thinking events are automatically converted to `ThinkingPart` in `UIMessage` objects. They are UI-only and excluded from messages sent back to the model.

## Connection Adapters

Expand Down
37 changes: 19 additions & 18 deletions docs/guides/tool-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ sequenceDiagram

Note over LLM Service: Model analyzes tools<br/>and decides to use one

LLM Service-->>Server: Stream chunks:<br/>tool_call, content, done
Server-->>Browser: Forward chunks via SSE/HTTP
Browser->>Browser: Parse chunks &<br/>update UI
LLM Service-->>Server: Stream AG-UI events:<br/>TOOL_CALL_*, TEXT_MESSAGE_*, RUN_FINISHED
Server-->>Browser: Forward events via SSE/HTTP
Browser->>Browser: Parse events &<br/>update UI
Browser->>User: Show response
```

Expand All @@ -57,11 +57,11 @@ sequenceDiagram
- Analyzes the conversation and available tools
- Decides whether to call a tool based on the user's request
- Generates tool calls with arguments
5. **Streaming Response**: The LLM streams back chunks:
- `tool_call` chunks with tool name and arguments
- `content` chunks with text responses
- `done` chunk when complete
6. **Client Updates**: The browser receives chunks and updates the UI in real-time
5. **Streaming Response**: The LLM streams back AG-UI events:
- `TOOL_CALL_START` / `TOOL_CALL_ARGS` / `TOOL_CALL_END` events for tool invocations
- `TEXT_MESSAGE_CONTENT` events for text responses
- `RUN_FINISHED` event when complete
6. **Client Updates**: The browser receives events and updates the UI in real-time

### Code Example

Expand Down Expand Up @@ -114,14 +114,14 @@ Tools progress through different states during their lifecycle. Understanding th

```mermaid
stateDiagram-v2
[*] --> AwaitingInput: tool_call received
AwaitingInput --> InputStreaming: partial arguments
[*] --> AwaitingInput: TOOL_CALL_START received
AwaitingInput --> InputStreaming: TOOL_CALL_ARGS (partial)
InputStreaming --> InputComplete: all arguments received
InputComplete --> ApprovalRequested: needsApproval=true
InputComplete --> Executing: needsApproval=false
ApprovalRequested --> Executing: user approves
ApprovalRequested --> Cancelled: user denies
Executing --> OutputAvailable: success
Executing --> OutputAvailable: TOOL_CALL_END (success)
Executing --> OutputError: error
OutputAvailable --> [*]
OutputError --> [*]
Expand Down Expand Up @@ -210,15 +210,16 @@ sequenceDiagram
participant LLM
participant Tool

LLM->>Server: tool_call: send_email
LLM->>Server: TOOL_CALL_START: send_email
LLM->>Server: TOOL_CALL_ARGS: {to, subject, body}
Server->>Server: Check needsApproval
Server->>Client: approval-requested chunk
Server->>Client: CUSTOM event (approval-requested)
Client->>Client: Show approval UI
User->>Client: Clicks "Approve"
Client->>Server: POST approval response
Server->>Tool: execute(args)
Tool-->>Server: result
Server->>LLM: tool_result
Server->>LLM: TOOL_CALL_END with result
LLM-->>Client: Generate response
```

Expand Down Expand Up @@ -325,13 +326,13 @@ The LLM can call multiple tools in parallel for efficiency:

```mermaid
graph TD
A[LLM decides to call 3 tools] --> B[tool_call index: 0]
A --> C[tool_call index: 1]
A --> D[tool_call index: 2]
A[LLM decides to call 3 tools] --> B[TOOL_CALL_START index: 0]
A --> C[TOOL_CALL_START index: 1]
A --> D[TOOL_CALL_START index: 2]
B --> E[Execute in parallel]
C --> E
D --> E
E --> F[Collect all results]
E --> F[Collect all TOOL_CALL_END results]
F --> G[Continue with results]
```

Expand Down
Loading
Loading