Skip to content

fix: tool call streaming#152

Merged
think-in-universe merged 3 commits into
mainfrom
fix/tool-call-streaming
Nov 6, 2025
Merged

fix: tool call streaming#152
think-in-universe merged 3 commits into
mainfrom
fix/tool-call-streaming

Conversation

@think-in-universe
Copy link
Copy Markdown
Contributor

Fixes #150

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces specialized delta types for streaming chat completions to properly handle incremental tool call responses. The change addresses the mismatch between complete ToolCall structures and the partial delta chunks received in streaming responses.

  • Changed ChatDelta.tool_calls from Vec<ToolCall> to Vec<ToolCallDelta>
  • Added new ToolCallDelta struct with optional fields and an index field for chunk ordering
  • Added new FunctionCallDelta struct with optional fields for partial function call data

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

#[serde(skip_serializing_if = "Option::is_none")]
pub id: Option<String>,
#[serde(rename = "type", skip_serializing_if = "Option::is_none")]
pub type_: Option<String>,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use r#type maybe better than type_, this field also exists in some other structs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can change this all together in another PR. I prefer to keep it consistent for now.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

crates/api/src/conversions.rs:93

  • The new ToolCalls finish reason lacks test coverage. The existing tests in this file only cover message and request conversions. Consider adding a test case for finish_reason_to_string that verifies all finish reason variants, including the new ToolCalls variant.
fn finish_reason_to_string(reason: &services::FinishReason) -> String {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@think-in-universe think-in-universe merged commit 7cd21da into main Nov 6, 2025
6 of 7 checks passed
@think-in-universe think-in-universe deleted the fix/tool-call-streaming branch November 6, 2025 06:47
@think-in-universe
Copy link
Copy Markdown
Contributor Author

think-in-universe commented Nov 6, 2025

@PierreLeGuen for some context, we have seen below errors when testing the tool call:

Failed to parse chat chunk: missing field `name` for json: {"choices":[{"delta":{"tool_calls":[{"function":{"arguments":"\"}"},"index":0}]},"finish_reason":null,"index":0,"logprobs":null,"token_ids":null}],"created":1762403773,"id":"chatcmpl-746463feebbf402086a89d59d66112ad","model":"deepseek-ai/DeepSeek-V3.1","object":"chat.completion.chunk"}

The name is required in FunctionCall:

/// Function call details
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FunctionCall {
    pub name: String,
    pub arguments: Option<String>,
}

Also, tool_calls type is needed for FinishReason:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum FinishReason {
    Stop,
    Length,
    ContentFilter,
    #[serde(alias = "function_call")]
    ToolCalls,
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: tool call with stream: true returns Invalid response format

3 participants