[None][perf] Validate Kimi-K2.5 transceiver v2 performance by Shixiaowei02 · Pull Request #15966 · NVIDIA/TensorRT-LLM

Shixiaowei02 · 2026-07-06T07:23:34Z

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

Tests
- Updated several performance sanity test configurations to use explicit cache and runtime settings.
- Standardized test setup across multiple GB200 and GB300 benchmark scenarios.
- Improved consistency of benchmark runs by removing reliance on implicit defaults.

Shixiaowei02 · 2026-07-06T07:26:15Z

/bot run --disable-fail-fast --stage-list "GB200PerfSanity*,GB300PerfSanity*"

coderabbitai · 2026-07-06T07:27:23Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: bf49873c-7e25-41d9-ba9f-3d491b37673d

📥 Commits

Reviewing files that changed from the base of the PR and between 3b5ce7c and 96efe46.

📒 Files selected for processing (12)

tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con2048_ctx1_dep4_gen1_dep32_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con2048_ctx1_dep4_gen1_dep32_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4096_ctx1_dep4_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_1k1k_con4_ctx1_dep4_gen1_tep4_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb416_mtp3_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp0_ccb-NIXL.yaml
tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_8k1k_con4_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml

📝 Walkthrough

Walkthrough

Fourteen disaggregated perf-sanity YAML test configuration files (GB200 and GB300 variants) are updated to explicitly add use_kv_cache_manager_v2: false under kv_cache_config and transceiver_runtime: PYTHON under cache_transceiver_config, applied consistently in both worker_config.gen and worker_config.ctx sections.

Changes

Perf-sanity YAML config updates

Layer / File(s)	Summary
GB200 config updates `tests/scripts/perf-sanity/disaggregated/gb200_kimi-k25-thinking-fp4_*.yaml`	Six GB200 configs add `use_kv_cache_manager_v2: false` and `transceiver_runtime: PYTHON` under `gen` and `ctx` worker_config sections.
GB300 config updates `tests/scripts/perf-sanity/disaggregated/gb300_kimi-k25-thinking-fp4_*.yaml`	Seven GB300 configs add the same `use_kv_cache_manager_v2: false` and `transceiver_runtime: PYTHON` settings under `gen` and `ctx` worker_config sections.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is just the template and does not explain the issue, solution, or test coverage.	Add a short Description of the change and a Test Coverage section with the relevant configs or validation steps.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title matches the PR's main goal of validating Kimi-K2.5 transceiver v2 performance.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

tensorrt-cicd · 2026-07-06T07:32:20Z

PR_Github #57724 [ run ] triggered by Bot. Commit: 588936c Link to invocation

tensorrt-cicd · 2026-07-06T11:52:34Z

PR_Github #57724 [ run ] completed with state SUCCESS. Commit: 588936c
/LLM/main/L0_MergeRequest_PR pipeline #46434 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-07-08T09:44:33Z

/bot run --disable-fail-fast --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE1-GPU4-Post-Merge"

tensorrt-cicd · 2026-07-08T09:51:02Z

PR_Github #58216 [ run ] triggered by Bot. Commit: 588936c Link to invocation

tensorrt-cicd · 2026-07-08T10:15:29Z

PR_Github #58216 [ run ] completed with state FAILURE. Commit: 588936c
/LLM/main/L0_MergeRequest_PR pipeline #46858 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Shixiaowei02 · 2026-07-08T10:35:54Z

/bot run --disable-fail-fast --stage-list "GB200-4_GPUs-PyTorch-PerfSanity-Post-Merge-7"

tensorrt-cicd · 2026-07-08T10:41:50Z

PR_Github #58224 [ run ] triggered by Bot. Commit: 588936c Link to invocation

tensorrt-cicd · 2026-07-08T11:06:18Z

PR_Github #58224 [ run ] completed with state SUCCESS. Commit: 588936c
/LLM/main/L0_MergeRequest_PR pipeline #46866 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

Shixiaowei02 · 2026-07-08T11:43:43Z

/bot run --disable-fail-fast --stage-list "GB200PerfSanity*,GB300PerfSanity*"

tensorrt-cicd · 2026-07-08T11:49:08Z

PR_Github #58228 [ run ] triggered by Bot. Commit: fc8ad13 Link to invocation

tensorrt-cicd · 2026-07-08T15:32:13Z

PR_Github #58228 [ run ] completed with state FAILURE. Commit: fc8ad13
/LLM/main/L0_MergeRequest_PR pipeline #46870 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Enable the NIXL transceiver v2 Python runtime (transceiver_runtime: PYTHON) for both context and generation workers across the Kimi-K2.5-thinking-fp4 disaggregated perf-sanity configs on GB200 and GB300, mirroring the GPT-OSS validation in PR NVIDIA#15765. KV cache manager v2 (the default) is kept enabled. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

Shixiaowei02 · 2026-07-09T07:51:58Z

/bot run --disable-fail-fast --stage-list "GB200PerfSanity*,GB300PerfSanity*"

tensorrt-cicd · 2026-07-09T07:58:44Z

PR_Github #58421 [ run ] triggered by Bot. Commit: a76a77a Link to invocation

tensorrt-cicd · 2026-07-09T12:17:46Z

PR_Github #58421 [ run ] completed with state FAILURE. Commit: a76a77a
/LLM/main/L0_MergeRequest_PR pipeline #47037 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

github-actions Bot assigned Shixiaowei02 Jul 6, 2026

Shixiaowei02 force-pushed the user/xiaoweis/kimi-perf branch from 96efe46 to 588936c Compare July 6, 2026 07:25

Shixiaowei02 marked this pull request as draft July 6, 2026 07:26

Shixiaowei02 force-pushed the user/xiaoweis/kimi-perf branch 2 times, most recently from 0837be9 to fc8ad13 Compare July 8, 2026 11:31

Shixiaowei02 marked this pull request as ready for review July 8, 2026 11:31

Shixiaowei02 requested review from chenfeiz0326, chuangz0 and nv-xtf July 8, 2026 11:31

Shixiaowei02 marked this pull request as draft July 8, 2026 11:38

Shixiaowei02 force-pushed the user/xiaoweis/kimi-perf branch from fc8ad13 to a76a77a Compare July 9, 2026 07:50

Uh oh!

Conversation

Shixiaowei02 commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

Summary by CodeRabbit

Uh oh!

Shixiaowei02 commented Jul 6, 2026

Uh oh!

coderabbitai Bot commented Jul 6, 2026

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

Shixiaowei02 commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Shixiaowei02 commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Shixiaowei02 commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Shixiaowei02 commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Shixiaowei02 commented Jul 6, 2026 •

edited

Loading