[None][feat] Support tool-calling in KvCacheAwareRouter for disagg serving (feat/deepseek_v4)

[lishicheng1996-nv]

Description

Cherry-pick of #13232 onto feat/deepseek_v4.

KvCacheAwareRouter._tokenize pre-tokenizes ChatCompletionRequests to compute routing block-hashes, and sets request.prompt_token_ids so the worker server skips re-tokenization. Today it only forwards request.messages to apply_chat_template, ignoring request.tools and request.chat_template_kwargs.

For models whose chat templates materialize tools into the prompt (DeepSeek-V3.2 inlines a tool-schema system block via its custom template), this silently drops the entire tool block from the final prompt — even though the client provided tools in the API request. Tool-calling requests are then run against a prompt that never saw the tools, which both degrades tool-use behavior and produces incorrect cache-aware routing decisions (block hashes computed over a truncated prompt).

Change

Forward request.tools (unwrapped via model_dump()) and request.chat_template_kwargs into apply_chat_template, so router-side pre-tokenization produces the same prompt token IDs the worker would have produced on its own — tools and template flags included. No behavior change for requests without tools or chat_template_kwargs.

Test Coverage

Reproduced on DeepSeek-V3.2 1P1D (DEP8 CTX + TEP8 GEN) with an agentic coding benchmark (12 tools per request, chat_template_kwargs={"thinking": true}):

• cache hit rate: 95.2% → 95.9%

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[lishicheng1996-nv]

/bot run --add-multi-gpu-test

[tensorrt-cicd]

PR_Github #50446 [ run ] triggered by Bot. Commit: 06adbed Link to invocation

[tensorrt-cicd]

PR_Github #50446 [ run ] completed with state SUCCESS. Commit: 06adbed
/LLM/main/L0_MergeRequest_PR pipeline #39965 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[lishicheng1996-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #50566 [ run ] triggered by Bot. Commit: 06adbed Link to invocation

[tensorrt-cicd]

PR_Github #50566 [ run ] completed with state SUCCESS. Commit: 06adbed
/LLM/main/L0_MergeRequest_PR pipeline #40067 completed with status: 'SUCCESS'

CI Report

Link to invocation