Skip to content

Conversation

taohui
Copy link
Contributor

@taohui taohui commented Sep 22, 2025

…ming output

Purpose

Problem

In non-streaming output mode, DeepSeekV31ToolParser fails to correctly parse multiple tool calls.
For example, given the input:

<|tool▁call▁begin|>get_current_time<|tool▁sep|>{}<|tool▁call▁end|>
<|tool▁call▁begin|>big_number_add<|tool▁sep|>{"a": 923, "b": 24922}<|tool▁call▁end|>

The parser incorrectly extracts:

  • function_name: "get_current_time<|tool▁sep|>{}<|tool▁call▁end|><|tool▁call▁begin|>big_number_add"
  • function_arguments: '{"a": 923, "b": 24922}'

This merges two separate tool calls into one, breaking downstream processing.

Solution

  • Update the regex and parsing logic to correctly identify each <|tool▁call▁begin|> ... <|tool▁call▁end|> block independently.
  • Ensure that each tool call is parsed into its own function_name and function_arguments, even when multiple tools appear consecutively.

Result

Now, the same input correctly parses into:

  1. function_name: "get_current_time", arguments: {}
  2. function_name: "big_number_add", arguments: {"a": 923, "b": 24922}

Test Plan

I have added test cases in test_deepseekv31_tool_parser.py specifically covering the issue described above.
The tests verify that multiple tool calls in non-streaming output are parsed correctly,
ensuring each tool's function name and arguments are extracted independently.

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

…ming output

Signed-off-by: taohui <taohui3@gmail.com>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively addresses a bug in DeepSeekV31ToolParser where multiple tool calls in non-streaming mode were parsed incorrectly. The change from a greedy (.*) to a non-greedy (.*?) quantifier in the regular expression is the correct and standard solution for this type of parsing issue. The addition of a new test file with specific unit tests for single and multiple tool calls is excellent, ensuring the fix is robust and preventing future regressions. The code is clean and the changes are well-scoped. Overall, this is a high-quality contribution.

Signed-off-by: taohui <taohui3@gmail.com>
Signed-off-by: taohui <taohui3@gmail.com>
Copy link
Collaborator

@chaunceyjiang chaunceyjiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks~

LGTM

/cc @aarnphm PTAL

@chaunceyjiang chaunceyjiang added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 23, 2025
@DarkLight1337 DarkLight1337 merged commit e18b714 into vllm-project:main Sep 24, 2025
43 checks passed
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
…in non-streaming output (#25405)

Signed-off-by: taohui <taohui3@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deepseek Related to DeepSeek models frontend ready ONLY add when PR is ready to merge/full CI is needed tool-calling
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants