feat: reasoning, web search, and thinking event support#62
feat: reasoning, web search, and thinking event support#62jpr5 merged 6 commits intoCopilotKit:mainfrom
Conversation
…d Claude thinking blocks
jpr5
left a comment
There was a problem hiding this comment.
Code Review — 7-Agent Standard CR
878 lines across 7 files reviewed by 7 specialized agents.
Overall the PR is well-structured — the data flows are correctly traced, output indices are managed properly, and backward compatibility is maintained. Two bugs and three critical test gaps need addressing before merge.
Bugs
1. webSearches silently dropped for Claude Messages API
handleMessages() in src/messages.ts passes response.reasoning but never response.webSearches. A fixture with webSearches served via /v1/messages silently drops them — zero warning, zero log, zero error.
A user who defines a fixture with webSearches and tests via the Responses API will see it work correctly, then switch to Claude Messages API and see the web searches vanish with no indication anything was omitted.
Fix (minimum): Log a warning in handleMessages when response.webSearches is present:
if (response.webSearches?.length) {
logger.warn("webSearches in fixture response are not supported for Claude Messages API — ignoring");
}Or add a fixture-loader validation warning that webSearches only applies to the Responses API.
2. reasoning: "" silently treated as no reasoning
Both buildTextStreamEvents and buildClaudeTextStreamEvents use if (reasoning) — empty string is falsy in JS, so reasoning: "" produces zero events. Meanwhile, fixture-loader.ts validates it as a legal string. Gap between validation and runtime.
Fix: Add a validation warning for reasoning: "":
if (response.reasoning === "") {
results.push({ severity: "warning", fixtureIndex: i, message: "reasoning is empty string — no reasoning events will be emitted" });
}Same pattern as existing content: "" validation.
Missing Test Coverage (critical gaps)
3. No stream-collapse tests for reasoning/webSearch extraction
collapseOpenAISSE gained reasoning delta accumulation + web search query extraction from response.output_item.done. collapseAnthropicSSE gained thinking_delta extraction. Zero tests cover any of this. These functions power record-and-replay — wrong extraction means silent data loss in recorded fixtures.
Need tests for:
collapseOpenAISSEwith reasoning summary events →result.reasoningcollapseOpenAISSEwith web searchoutput_item.doneevents →result.webSearchescollapseAnthropicSSEwiththinking_deltaevents →result.reasoning
4. No fixture-loader validation tests
27 lines of new validation logic (reasoning type check, webSearches array check, element type check) with zero tests.
Need tests for:
reasoning: 123→ errorwebSearches: "not-array"→ errorwebSearches: ["valid", 42]→ error on element- Valid values → no errors
5. No WebSocket reasoning passthrough test
ws-responses.ts was modified to pass reasoning/webSearches through to buildTextStreamEvents — untested plumbing.
PR Description Fix
The PR body lists src/index.ts as changed with "Removed unexported builders from public API" — but index.ts is not in the diff. buildTextStreamEvents is still exported. Please update the description.
Non-blocking Notes
buildReasoningStreamEventshardcodesoutput_index: 0whilebuildWebSearchStreamEventstakesstartOutputIndex— asymmetric but correct today since reasoning is always firstwebSearches: string[]could beArray<{ query: string }>for extensibility — noted for future- Consider
webSearches: []andwebSearches: [""]validation warnings (empty array / empty string elements)
…ning; warn on webSearches in Claude handler
…ructural event skip
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
commit: |
## Summary Adds support for reasoning and web search events in fixture responses across multiple providers: - **OpenAI Responses API**: Optional `reasoning` field on `TextResponse` emits `response.reasoning_summary_text.delta` events before text content. Optional `webSearches` field emits `response.web_search_call` events. - **Anthropic Claude**: `reasoning` field emits `thinking` content blocks before text in `/v1/messages`. - Both HTTP SSE and WebSocket transports supported. - Backward compatible — existing fixtures work unchanged. Closes #60 ## Changes - **`src/types.ts`**: `TextResponse` extended with optional `reasoning?: string` and `webSearches?: string[]` - **`src/responses.ts`**: `buildTextStreamEvents` accepts optional reasoning/webSearches and prepends the appropriate events with correct `output_index` adjustment. Private helpers `buildReasoningStreamEvents` and `buildWebSearchStreamEvents` handle individual event sequences. Non-streaming `buildTextResponse` includes reasoning and web search output items. - **`src/messages.ts`**: Emits `thinking` content blocks when `reasoning` is present (both streaming and non-streaming). Warns when `webSearches` is used (not supported by Claude API). - **`src/ws-responses.ts`**: Passes reasoning/webSearches through to `buildTextStreamEvents` (no duplicated logic) - **`src/fixture-loader.ts`**: Validates `reasoning` (string) and `webSearches` (array of strings). Warns on empty reasoning. - **`src/stream-collapse.ts`**: Extracts reasoning from `response.reasoning_summary_text.delta`, text from `response.output_text.delta`, and web searches from `response.output_item.done` web_search_call events. Claude collapse extracts reasoning from `thinking_delta` events. ## Example ```typescript mock.addFixture({ match: { userMessage: "hello" }, response: { content: "Based on my research, here's what I found.", reasoning: "Let me think about what the user is asking...", webSearches: ["latest greeting conventions 2025"], }, }); ``` ## Test plan - [x] Reasoning events emitted correctly in Responses API SSE - [x] Reasoning deltas reconstruct full text - [x] Web search events emitted before text events - [x] Web search items contain query strings - [x] Combined reasoning + web search + text streaming - [x] Non-streaming responses include reasoning and web search output items - [x] Anthropic thinking blocks (streaming and non-streaming) - [x] Backward compatibility — plain text fixtures unchanged - [x] WebSocket reasoning passthrough - [x] Fixture-loader validation (reasoning type, empty string, webSearches type/elements) - [x] Stream-collapse extraction (OpenAI reasoning/text/web search, Anthropic thinking) - [x] All 1333 tests pass - [x] Prettier, ESLint clean
Summary
Adds support for reasoning and web search events in fixture responses across multiple providers:
reasoningfield onTextResponseemitsresponse.reasoning_summary_text.deltaevents before text content. OptionalwebSearchesfield emitsresponse.web_search_callevents.reasoningfield emitsthinkingcontent blocks before text in/v1/messages.Closes #60
Changes
src/types.ts:TextResponseextended with optionalreasoning?: stringandwebSearches?: string[]src/responses.ts:buildTextStreamEventsaccepts optional reasoning/webSearches and prepends the appropriate events with correctoutput_indexadjustment. Private helpersbuildReasoningStreamEventsandbuildWebSearchStreamEventshandle individual event sequences. Non-streamingbuildTextResponseincludes reasoning and web search output items.src/messages.ts: Emitsthinkingcontent blocks whenreasoningis present (both streaming and non-streaming). Warns whenwebSearchesis used (not supported by Claude API).src/ws-responses.ts: Passes reasoning/webSearches through tobuildTextStreamEvents(no duplicated logic)src/fixture-loader.ts: Validatesreasoning(string) andwebSearches(array of strings). Warns on empty reasoning.src/stream-collapse.ts: Extracts reasoning fromresponse.reasoning_summary_text.delta, text fromresponse.output_text.delta, and web searches fromresponse.output_item.doneweb_search_call events. Claude collapse extracts reasoning fromthinking_deltaevents.Example
Test plan