fix(llm, anthropic): Enable multiple tool calls in single turn#268
Merged
fix(llm, anthropic): Enable multiple tool calls in single turn#268
Conversation
Instructions containing XML tags, CDATA sequences, or other special characters now serialize without escaping special characters. This improves readability of instructions in the generated XML. The fix switches to a (temporary) `quick-xml` fork with CDATA serialization support and rewrites the XML serialization logic to wrap text content in CDATA sections. This ensures instruction text is preserved literally regardless of special characters. Additional formatting changes include proper nesting of items and examples, and setting the title as an attribute of the root element. The end-goal is better adherence to the instructions by LLM models. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Previously, when Anthropic models requested multiple tool calls during a single turn, the system would group multiple tool call argument JSON objects into a single tool call, causing argument parsing to fail. This occurred because only the MessageDelta event with stop_reason: `tool_use` was handled for tool call completion, but individual tool calls in a multi-call response generate separate `ContentBlockStop` events. The fix adds handling for `ContentBlockStop` events when the provider is accumulating tool call content, ensuring each tool call is properly finalized before the next one begins. This prevents JSON object concatenation and allows multiple tool calls to execute successfully in sequence, significantly speeding up tool call completion, and reducing token usage. Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, when Anthropic models requested multiple tool calls during a single turn, the system would group multiple tool call argument JSON objects into a single tool call, causing argument parsing to fail. This occurred because only the MessageDelta event with stop_reason:
tool_usewas handled for tool call completion, but individual tool calls in a multi-call response generate separateContentBlockStopevents.The fix adds handling for
ContentBlockStopevents when the provider is accumulating tool call content, ensuring each tool call is properly finalized before the next one begins. This prevents JSON object concatenation and allows multiple tool calls to execute successfully in sequence, significantly speeding up tool call completion, and reducing token usage.