[https://nvbugs/6248769][fix] AutoDeploy: clamp KV pool window to max_seq_len

[MrGeva]

Mixed sliding-window / full-attention models (e.g. mistralai/Ministral-8B-Instruct-2410) report a full-attention sliding_window larger than max_seq_len. In CachedSequenceInterface._identify_managed_kv_resources, the effective window was used unclamped as the KV pool key, producing two distinct pools (e.g. 512 and 32768 under max_seq_len=512). With the TRTLLM attention backend (requires_uniform_kv_caches=True) this tripped RuntimeError: KV resources are not uniform ... requires all V caches to share a single pool and failed the engine build.
This clamps the effective window to max_seq_len so windows that reach or exceed max_seq_len collapse into a single concrete pool key. Safe because at most max_seq_len tokens are ever cached.

Summary by CodeRabbit

Bug Fixes

• Fixed KV cache allocation to properly respect sequence length limits across all layers, preventing misaligned cache configurations.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[MrGeva]

/bot run

[coderabbitai]

📝 Walkthrough

Walkthrough

The change modifies CachedSequenceInterface._identify_managed_kv_resources() to clamp each KV layer's computed effective_window to max_seq_len. This ensures that sliding-window layers with values larger than max_seq_len collapse into a single pool key, maintaining uniform KV cache allocation across layers.

Changes

KV Cache Resource Management

Layer / File(s)	Summary
Effective window clamping in KV resource identification tensorrt_llm/_torch/auto_deploy/shim/interface.py	_identify_managed_kv_resources() clamps effective_window to max_seq_len for each KV layer, preventing layers with sliding windows larger than the maximum sequence length from producing distinct pool keys.

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The PR description clearly explains the issue (mixed sliding-window models producing non-uniform KV pools) and the solution (clamping effective window to max_seq_len), with justification for safety. However, the author did not fill in the Description, Test Coverage, or PR Checklist sections in the template.	Complete the PR description template by explicitly filling in the Description and Test Coverage sections, even if referencing the issue discussion or existing CI tests. Update the PR Checklist (currently all unchecked except the final statement) to document which items were actually reviewed.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: clamping KV pool window to max_seq_len in AutoDeploy, with a specific NVBugs reference and [fix] type tag.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[tensorrt-cicd]

PR_Github #51837 [ run ] triggered by Bot. Commit: a6bf30e Link to invocation

[MrGeva]

/bot run

[tensorrt-cicd]

PR_Github #51839 [ run ] triggered by Bot. Commit: 6475266 Link to invocation

[tensorrt-cicd]

PR_Github #51837 [ run ] completed with state ABORTED. Commit: a6bf30e

Link to invocation

[tensorrt-cicd]

PR_Github #51839 [ run ] completed with state SUCCESS. Commit: 6475266
/LLM/main/L0_MergeRequest_PR pipeline #41201 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[tburt-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #51895 [ run ] triggered by Bot. Commit: 6475266 Link to invocation

[tensorrt-cicd]

PR_Github #51895 [ run ] completed with state SUCCESS. Commit: 6475266
/LLM/main/L0_MergeRequest_PR pipeline #41250 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation