[None][perf] Validate GPT-OSS transceiver v2 performance#15765
Conversation
|
/bot run --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4" |
|
PR_Github #56561 [ run ] triggered by Bot. Commit: |
|
PR_Github #56561 [ run ] completed with state |
42b8d28 to
6ccaf61
Compare
📝 WalkthroughWalkthroughSeven disaggregated perf-sanity benchmark YAML configuration files are updated to explicitly set ChangesDisaggregated perf-sanity YAML config updates
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot run --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57139 [ run ] triggered by Bot. Commit: |
|
PR_Github #57139 [ run ] completed with state
|
|
/bot run --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-6,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57157 [ run ] triggered by Bot. Commit: |
|
PR_Github #57157 [ run ] completed with state
|
|
/bot run --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-6,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57306 [ run ] triggered by Bot. Commit: |
|
PR_Github #57306 [ run ] completed with state
|
6ccaf61 to
7a77462
Compare
|
/bot run --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-6,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57320 [ run ] triggered by Bot. Commit: |
|
PR_Github #57320 [ run ] completed with state
|
Signed-off-by: Tingfeng Xian <289617005+nv-xtf@users.noreply.github.com>
…iver v2 Signed-off-by: Tingfeng Xian <289617005+nv-xtf@users.noreply.github.com>
7a77462 to
6199434
Compare
Enable KV cache manager v1 (use_kv_cache_manager_v2: false) and NIXL transceiver v2 Python runtime (transceiver_runtime: PYTHON) for both context and generation workers across the Kimi-K2.5-thinking-fp4 disaggregated perf-sanity configs on GB200 and GB300, mirroring the GPT-OSS validation in PR NVIDIA#15765. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
…iver v2 Signed-off-by: Tingfeng Xian <289617005+nv-xtf@users.noreply.github.com>
|
/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-1,DGX_B200-4_GPUs-PyTorch-2,DGX_B200-4_GPUs-PyTorch-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-" |
|
PR_Github #57728 [ run ] triggered by Bot. Commit: |
|
PR_Github #57728 [ run ] completed with state
|
|
/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-1,DGX_B200-4_GPUs-PyTorch-2,DGX_B200-4_GPUs-PyTorch-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-" |
|
PR_Github #57885 [ run ] triggered by Bot. Commit: |
|
PR_Github #57885 [ run ] completed with state
|
|
/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-1,DGX_B200-4_GPUs-PyTorch-2,DGX_B200-4_GPUs-PyTorch-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-6,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57894 [ run ] triggered by Bot. Commit: |
|
/bot kill |
|
PR_Github #57953 [ kill ] triggered by Bot. Commit: |
|
PR_Github #57894 [ run ] completed with state |
|
PR_Github #57953 [ kill ] completed with state |
|
/bot run --stage-list "DGX_B200-4_GPUs-PyTorch-1,DGX_B200-4_GPUs-PyTorch-2,DGX_B200-4_GPUs-PyTorch-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU2-Post-Merge-4,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-1,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-2,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-3,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-6,GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU1-GEN1-NODE1-GPU4-Post-Merge-7" |
|
PR_Github #57959 [ run ] triggered by Bot. Commit: |
|
PR_Github #57959 [ run ] completed with state |
|
One question on the perf disagg configs before this merges. Under
The net effect is that after this PR the gpt-oss perf coverage is split: the NIXL cases move to the V2 / PYTHON transceiver, while these two UCX cases stay on CPP / KV cache v1. Is keeping these UCX variants intentional (we still want CPP/UCX perf coverage for gpt-oss), or should they be removed so gpt-oss perf standardizes on the NIXL + PYTHON path? Just want to confirm this split is deliberate and not an oversight. |
|
/bot skip --comment "Targeted GPT-OSS Harmony and perf-sanity tests passed." |
|
PR_Github #57977 [ skip ] triggered by Bot. Commit: |
|
PR_Github #57977 [ skip ] completed with state |
Enable the NIXL transceiver v2 Python runtime (transceiver_runtime: PYTHON) for both context and generation workers across the Kimi-K2.5-thinking-fp4 disaggregated perf-sanity configs on GB200 and GB300, mirroring the GPT-OSS validation in PR NVIDIA#15765. KV cache manager v2 (the default) is kept enabled. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
Enable the NIXL transceiver v2 Python runtime (transceiver_runtime: PYTHON) for both context and generation workers across the Kimi-K2.5-thinking-fp4 disaggregated perf-sanity configs on GB200 and GB300, mirroring the GPT-OSS validation in PR NVIDIA#15765. KV cache manager v2 (the default) is kept enabled. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
Signed-off-by: Tingfeng Xian <289617005+nv-xtf@users.noreply.github.com>
Enable the NIXL transceiver v2 Python runtime (transceiver_runtime: PYTHON) for both context and generation workers across the Kimi-K2.5-thinking-fp4 disaggregated perf-sanity configs on GB200 and GB300, mirroring the GPT-OSS validation in PR NVIDIA#15765. KV cache manager v2 (the default) is kept enabled. Signed-off-by: Shixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>
Description
Migrate existing NIXL-based GPT-OSS disaggregated tests from Transceiver V1 to V2 (
transceiver_runtime: PYTHON).This PR updates the accuracy, stress, and performance test configurations. Performance tests continue to use KV cache manager v1. A separate V1 test lane is not retained.
Performance Validation
Run the existing GB200 GPT-OSS perf-sanity cases and compare the V2 pre-merge results with the historical V1 post-merge baseline.
Performance results
1k1k con64result matches the post-merge level observed after itsJune 29 unwaive, indicating that its stored baseline is stale.
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.Summary by CodeRabbit