Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/think-send-reasoning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cloudflare/think": patch
---

Add `sendReasoning` controls to Think. Subclasses can set an instance-wide default, and `beforeTurn` can return a per-turn override to include or suppress reasoning chunks in UI message streams.
1 change: 1 addition & 0 deletions docs/think/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ Both Think and [`AIChatAgent`](../chat-agents.md) extend `Agent` and speak the s
| `getSystemPrompt()` | `"You are a helpful assistant."` | System prompt (fallback when no context blocks) |
| `getTools()` | `{}` | AI SDK `ToolSet` for the agentic loop |
| `maxSteps` | `10` | Max tool-call rounds per turn |
| `sendReasoning` | `true` | Send reasoning chunks to chat clients |
| `configureSession()` | identity | Add context blocks, compaction, search, skills — see [Sessions](../sessions.md) |
| `messageConcurrency` | `"queue"` | How overlapping submits behave — see [Client Tools](./client-tools.md) |
| `waitForMcpConnections` | `false` | Wait for MCP servers before inference |
Expand Down
11 changes: 11 additions & 0 deletions docs/think/lifecycle-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ All fields are optional. Return only what you want to change.
| `activeTools` | `string[]` | Limit which tools the model can call |
| `toolChoice` | `ToolChoice` | Force a specific tool call |
| `maxSteps` | `number` | Override `maxSteps` for this turn |
| `sendReasoning` | `boolean` | Send reasoning chunks for this turn |
| `providerOptions` | `Record<string, unknown>` | Provider-specific options |

### Examples
Expand Down Expand Up @@ -169,6 +170,16 @@ beforeTurn(ctx: TurnContext) {
}
```

Hide reasoning for internal continuation turns:

```typescript
beforeTurn(ctx: TurnContext) {
if (ctx.continuation) {
return { sendReasoning: false };
}
}
```

Force structured output for a turn (Vercel AI SDK `Output.object`). Combine with `activeTools: []` because some providers (e.g. `workers-ai-provider`) strip tools when `responseFormat: "json"` is active:

```typescript
Expand Down
5 changes: 5 additions & 0 deletions packages/think/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ export class MyAgent extends Think<Env> {
| `getSystemPrompt()` | `"You are a helpful assistant."` | System prompt (fallback when no context blocks) |
| `getTools()` | `{}` | AI SDK `ToolSet` for the agentic loop |
| `maxSteps` | `10` | Max tool-call rounds per turn (property) |
| `sendReasoning` | `true` | Send reasoning chunks to chat clients |
| `configureSession()` | identity | Add context blocks, compaction, search, skills |
| `getExtensions()` | `[]` | Sandboxed extension declarations (load order) |
| `extensionLoader` | `undefined` | `WorkerLoader` binding — enables extensions |
Expand Down Expand Up @@ -138,6 +139,8 @@ The AI SDK-derived contexts spread the SDK's own types at the top level — no i

`beforeStep` is wired to the AI SDK's `prepareStep` callback. Return a `StepConfig` to override `model`, `toolChoice`, `activeTools`, `system`, `messages`, `experimental_context`, or `providerOptions` for the current step. The AI SDK does not expose `output` or `maxSteps` per step — set those at the turn level via `TurnConfig` (returned from `beforeTurn`). `beforeStep` is subclass-only; it is not dispatched to extensions because the prepareStep event surface includes a live `LanguageModel` instance which is not JSON-safe to snapshot.

`TurnConfig` also accepts `sendReasoning` to override whether reasoning chunks are emitted for the current UI message stream. The instance-level `sendReasoning` property defaults to `true`; return `{ sendReasoning: false }` from `beforeTurn` to hide reasoning for a single turn, for example on internal continuation turns.

`TurnConfig` also accepts an `output` field that is forwarded to `streamText` as the AI SDK's structured-output spec. Combine with `activeTools: []` for providers (e.g. `workers-ai-provider`) that strip tools when `responseFormat: "json"` is active. Use `experimental_telemetry` to pass the AI SDK's per-call telemetry settings through to `streamText`; consider disabling `recordInputs` or `recordOutputs` if prompts or outputs may contain sensitive data.

Per-tool hooks are wired so `beforeToolCall` fires _before_ `execute` (Think wraps every tool's `execute`) and `afterToolCall` fires _after_ (via the AI SDK's `experimental_onToolCallFinish`) with `durationMs` and a discriminated outcome. `beforeToolCall` can return a `ToolCallDecision` to:
Expand Down Expand Up @@ -253,7 +256,9 @@ interface TurnConfig {
activeTools?: string[]; // limit which tools the model can call
toolChoice?: ToolChoice; // force a specific tool
maxSteps?: number; // override maxSteps for this turn
sendReasoning?: boolean; // send reasoning chunks for this turn
providerOptions?: Record<string, unknown>;
experimental_telemetry?: TelemetrySettings;
}
```

Expand Down
64 changes: 64 additions & 0 deletions packages/think/src/tests/agents/think-session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,51 @@ function createMockModel(response: string): LanguageModel {
} as LanguageModel;
}

function createReasoningMockModel(
response: string,
reasoning: string
): LanguageModel {
return {
specificationVersion: "v3",
provider: "test",
modelId: "mock-reasoning-model",
supportedUrls: {},
doGenerate() {
throw new Error("doGenerate not implemented in mock");
},
doStream() {
_mockCallCount++;
const callId = _mockCallCount;
const stream = new ReadableStream({
start(controller) {
controller.enqueue({ type: "stream-start", warnings: [] });
controller.enqueue({ type: "reasoning-start", id: `r-${callId}` });
controller.enqueue({
type: "reasoning-delta",
id: `r-${callId}`,
delta: reasoning
});
controller.enqueue({ type: "reasoning-end", id: `r-${callId}` });
controller.enqueue({ type: "text-start", id: `t-${callId}` });
controller.enqueue({
type: "text-delta",
id: `t-${callId}`,
delta: response
});
controller.enqueue({ type: "text-end", id: `t-${callId}` });
controller.enqueue({
type: "finish",
finishReason: v3FinishReason("stop"),
usage: v3Usage(10, 8)
});
controller.close();
}
});
return Promise.resolve({ stream });
}
} as LanguageModel;
}

/** Mock model that emits multiple text-delta chunks for abort testing */
function createMultiChunkMockModel(chunks: string[]): LanguageModel {
return {
Expand Down Expand Up @@ -248,6 +293,8 @@ export class ThinkTestAgent extends Think {
private _stepConfigOverride: StepConfig | null = null;
private _beforeStepAsyncDelayMs = 0;
private _telemetryEvents: string[] = [];
private _reasoningResponse: { response: string; reasoning: string } | null =
null;
private _beforeStepLog: Array<{
stepNumber: number;
previousStepCount: number;
Expand All @@ -273,6 +320,10 @@ export class ThinkTestAgent extends Think {
this._turnConfigOverride = config;
}

async setSendReasoningDefault(sendReasoning: boolean): Promise<void> {
this.sendReasoning = sendReasoning;
}

/**
* Set a `TurnConfig.output` override using the AI SDK's `Output.text()`
* helper. The Output spec contains promises and other non-cloneable
Expand Down Expand Up @@ -526,7 +577,20 @@ export class ThinkTestAgent extends Think {
this._multiChunks = null;
}

async setReasoningResponse(
response: string,
reasoning: string
): Promise<void> {
this._reasoningResponse = { response, reasoning };
}

override getModel(): LanguageModel {
if (this._reasoningResponse) {
return createReasoningMockModel(
this._reasoningResponse.response,
this._reasoningResponse.reasoning
);
}
if (this._multiChunks) {
return createMultiChunkMockModel(this._multiChunks);
}
Expand Down
163 changes: 162 additions & 1 deletion packages/think/src/tests/hooks.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,99 @@
import { describe, expect, it } from "vitest";
import { env } from "cloudflare:workers";
import { env, exports } from "cloudflare:workers";
import { getAgentByName } from "agents";
import type { UIMessage } from "ai";

const MSG_CHAT_REQUEST = "cf_agent_use_chat_request";
const MSG_CHAT_RESPONSE = "cf_agent_use_chat_response";

async function connectWS(agentClass: string, room: string) {
const slug = agentClass
.replace(/([a-z])([A-Z])/g, "$1-$2")
.replace(/([A-Z]+)([A-Z][a-z])/g, "$1-$2")
.toLowerCase();
const res = await exports.default.fetch(
`http://example.com/agents/${slug}/${room}`,
{ headers: { Upgrade: "websocket" } }
);
expect(res.status).toBe(101);
const ws = res.webSocket as WebSocket;
expect(ws).toBeDefined();
ws.accept();
return ws;
}

function waitForDone(
ws: WebSocket,
timeout = 10000
): Promise<Array<Record<string, unknown>>> {
return new Promise((resolve, reject) => {
const messages: Array<Record<string, unknown>> = [];
const timer = setTimeout(
() => reject(new Error("Timeout waiting for done")),
timeout
);
const handler = (e: MessageEvent) => {
try {
const msg = JSON.parse(e.data as string) as Record<string, unknown>;
messages.push(msg);
if (msg.type === MSG_CHAT_RESPONSE && msg.done === true) {
clearTimeout(timer);
ws.removeEventListener("message", handler);
resolve(messages);
}
} catch {
// ignore non-JSON frames
}
};
ws.addEventListener("message", handler);
});
}

function closeWS(ws: WebSocket): Promise<void> {
return new Promise((resolve) => {
const timer = setTimeout(resolve, 200);
ws.addEventListener(
"close",
() => {
clearTimeout(timer);
resolve();
},
{ once: true }
);
ws.close();
});
}

function sendChatRequest(ws: WebSocket, text: string) {
const userMessage: UIMessage = {
id: crypto.randomUUID(),
role: "user",
parts: [{ type: "text", text }]
};
ws.send(
JSON.stringify({
type: MSG_CHAT_REQUEST,
id: crypto.randomUUID(),
init: {
method: "POST",
body: JSON.stringify({ messages: [userMessage] })
}
})
);
}

function eventTypes(events: string[]): string[] {
return events.map((event) => (JSON.parse(event) as { type: string }).type);
}

function websocketChunkTypes(
messages: Array<Record<string, unknown>>
): string[] {
return messages
.filter((msg) => msg.type === MSG_CHAT_RESPONSE && msg.done === false)
.map((msg) => JSON.parse(msg.body as string) as { type: string })
.map((chunk) => chunk.type);
}

async function freshAgent(name: string) {
return getAgentByName(env.ThinkTestAgent, name);
Expand Down Expand Up @@ -726,4 +819,72 @@ describe("Think — beforeTurn config overrides", () => {
const result = await agent.testChat("Structured-output turn");
expect(result.done).toBe(true);
});

it("sends reasoning chunks by default on the chat() path", async () => {
const agent = await freshAgent("bt-reasoning-default");
await agent.setReasoningResponse("Final answer", "Visible thinking");

const result = await agent.testChat("Show reasoning");
const types = eventTypes(result.events);

expect(types).toContain("reasoning-start");
expect(types).toContain("reasoning-delta");
expect(types).toContain("reasoning-end");
});

it("uses the instance-level sendReasoning default", async () => {
const agent = await freshAgent("bt-reasoning-instance");
await agent.setSendReasoningDefault(false);
await agent.setReasoningResponse("Final answer", "Hidden thinking");

const result = await agent.testChat("Hide reasoning");
const types = eventTypes(result.events);

expect(types).not.toContain("reasoning-start");
expect(types).not.toContain("reasoning-delta");
expect(types).not.toContain("reasoning-end");
expect(types).toContain("text-delta");
});

it("allows TurnConfig to suppress reasoning for one turn", async () => {
const agent = await freshAgent("bt-reasoning-turn-false");
await agent.setReasoningResponse("Final answer", "Hidden thinking");
await agent.setTurnConfigOverride({ sendReasoning: false });

const result = await agent.testChat("Hide reasoning this turn");
const types = eventTypes(result.events);

expect(types).not.toContain("reasoning-delta");
expect(types).toContain("text-delta");
});

it("allows TurnConfig to send reasoning when the instance default is false", async () => {
const agent = await freshAgent("bt-reasoning-turn-true");
await agent.setSendReasoningDefault(false);
await agent.setReasoningResponse("Final answer", "Visible thinking");
await agent.setTurnConfigOverride({ sendReasoning: true });

const result = await agent.testChat("Show reasoning this turn");
const types = eventTypes(result.events);

expect(types).toContain("reasoning-delta");
expect(types).toContain("text-delta");
});

it("applies sendReasoning on the WebSocket stream path", async () => {
const room = "bt-reasoning-ws";
const agent = await freshAgent(room);
await agent.setTurnConfigOverride({ sendReasoning: false });
await agent.setReasoningResponse("Final answer", "Hidden thinking");

const ws = await connectWS("ThinkTestAgent", room);
const done = waitForDone(ws);
sendChatRequest(ws, "Hide reasoning over WebSocket");
const messages = await done;
await closeWS(ws);

const types = websocketChunkTypes(messages);
expect(types).not.toContain("reasoning-delta");
expect(types).toContain("text-delta");
});
});
25 changes: 23 additions & 2 deletions packages/think/src/think.ts
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,9 @@ export interface StreamCallback {
* The AI SDK's `streamText()` result satisfies this interface.
*/
export interface StreamableResult {
toUIMessageStream(): AsyncIterable<unknown>;
toUIMessageStream(options?: {
sendReasoning?: boolean;
}): AsyncIterable<unknown>;
}

/**
Expand Down Expand Up @@ -307,6 +309,11 @@ export interface TurnConfig {
toolChoice?: Parameters<typeof streamText>[0]["toolChoice"];
/** Override maxSteps for this turn. */
maxSteps?: number;
/**
* Controls whether reasoning chunks are included in the UI message stream
* for this turn. Defaults to the instance-level `sendReasoning` setting.
*/
sendReasoning?: boolean;
/** Provider-specific options (AI SDK providerOptions). */
providerOptions?: Record<string, unknown>;
/** Optional AI SDK telemetry configuration for this turn. */
Expand Down Expand Up @@ -841,6 +848,12 @@ export class Think<
/** Maximum number of tool-call steps per turn. Override via property or per-turn via TurnConfig. */
maxSteps = 10;

/**
* Whether reasoning chunks are sent to chat clients by default. Override
* per turn by returning `sendReasoning` from `beforeTurn`.
*/
sendReasoning = true;

/**
* Configure the session. Called once during `onStart`.
* Override to add context blocks, compaction, search, skills.
Expand Down Expand Up @@ -1191,6 +1204,7 @@ export class Think<
// (optionally with modified `input`).
const finalTools: ToolSet = this._wrapToolsWithDecision(mergedTools);
const finalMaxSteps = config.maxSteps ?? this.maxSteps;
const finalSendReasoning = config.sendReasoning ?? this.sendReasoning;

const result = streamText({
model: finalModel,
Expand Down Expand Up @@ -1266,7 +1280,12 @@ export class Think<
}) satisfies StreamTextOnToolCallFinishCallback<ToolSet>
});

return this._transformInferenceResult(result);
const streamResult = {
toUIMessageStream: () =>
result.toUIMessageStream({ sendReasoning: finalSendReasoning })
} satisfies StreamableResult;

return this._transformInferenceResult(streamResult);
}

/** @internal Test seam — override in test agents to wrap the stream (e.g. error injection). */
Expand Down Expand Up @@ -1336,6 +1355,8 @@ export class Think<
accumulated.toolChoice = parsed.config.toolChoice;
if (parsed.config.maxSteps !== undefined)
accumulated.maxSteps = parsed.config.maxSteps;
if (parsed.config.sendReasoning !== undefined)
accumulated.sendReasoning = parsed.config.sendReasoning;
if (parsed.config.providerOptions !== undefined) {
accumulated.providerOptions = {
...(accumulated.providerOptions ?? {}),
Expand Down
Loading