fix: tool call streaming#152
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR introduces specialized delta types for streaming chat completions to properly handle incremental tool call responses. The change addresses the mismatch between complete ToolCall structures and the partial delta chunks received in streaming responses.
- Changed
ChatDelta.tool_callsfromVec<ToolCall>toVec<ToolCallDelta> - Added new
ToolCallDeltastruct with optional fields and anindexfield for chunk ordering - Added new
FunctionCallDeltastruct with optional fields for partial function call data
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #[serde(skip_serializing_if = "Option::is_none")] | ||
| pub id: Option<String>, | ||
| #[serde(rename = "type", skip_serializing_if = "Option::is_none")] | ||
| pub type_: Option<String>, |
There was a problem hiding this comment.
use r#type maybe better than type_, this field also exists in some other structs
There was a problem hiding this comment.
We can change this all together in another PR. I prefer to keep it consistent for now.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
crates/api/src/conversions.rs:93
- The new
ToolCallsfinish reason lacks test coverage. The existing tests in this file only cover message and request conversions. Consider adding a test case forfinish_reason_to_stringthat verifies all finish reason variants, including the newToolCallsvariant.
fn finish_reason_to_string(reason: &services::FinishReason) -> String {
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@PierreLeGuen for some context, we have seen below errors when testing the tool call:
The /// Function call details
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FunctionCall {
pub name: String,
pub arguments: Option<String>,
}Also, #[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum FinishReason {
Stop,
Length,
ContentFilter,
#[serde(alias = "function_call")]
ToolCalls,
} |
Fixes #150