[None][feat] add gemma and glm disagg python transceiver test by chuangz0 · Pull Request #15755 · NVIDIA/TensorRT-LLM

chuangz0 · 2026-06-30T04:29:31Z

Summary by CodeRabbit

New Features
- Added accuracy validation coverage for a new NVFP4 model configuration on disaggregated serving.
- Expanded cache manager testing to cover both supported cache modes in more scenarios.
Tests
- Updated test selections to include the new NVFP4 accuracy case and the revised cache manager variants.
- Adjusted GPU-specific test routing for the latest disaggregated serving coverage.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

chuangz0 · 2026-06-30T04:29:51Z

/bot run --disable-fail-fast

coderabbitai · 2026-06-30T04:30:40Z

📝 Walkthrough

Walkthrough

Adds a TestGLM5FP4 test class for NVFP4 disaggregated serving accuracy against GSM8K with a corresponding reference entry (accuracy: 78.0). Converts TestGemma3_1BInstruct.test_kv_cache_v2_nixl_python from a hardcoded v2 to a parametrized test covering both KV cache manager versions. Updates QA and hardware test lists accordingly.

Changes

GLM-5 FP4 test and Gemma3 parametrization

Layer / File(s)	Summary
GLM-5 NVFP4 accuracy reference and test class `tests/integration/defs/accuracy/references/gsm8k.yaml`, `tests/integration/defs/accuracy/test_disaggregated_serving.py`	Adds GSM8K reference entry for `zai-org/GLM-5` with `quant_algo: NVFP4` and `accuracy: 78.0`. Introduces `TestGLM5FP4` class with `test_nvfp4_nixl_python`, parameterized to `cache_mgr_v1` only, launching NIXL Python disaggregated serving and running GSM8K accuracy.
Gemma3 KV cache test parametrization `tests/integration/defs/accuracy/test_disaggregated_serving.py`	Converts `test_kv_cache_v2_nixl_python` from hardcoded `use_kv_cache_manager_v2=True` to a `@pytest.mark.parametrize` covering both v1 and v2, wired into context and generation server configs.
Test list updates `tests/integration/test_lists/qa/llm_function_core.txt`, `tests/integration/test_lists/test-db/l0_dgx_b200.yml`, `tests/integration/test_lists/test-db/l0_dgx_b300.yml`, `tests/integration/test_lists/test-db/l0_dgx_h100.yml`	Adds `TestGLM5FP4` to B200 and QA lists, expands Gemma3 H100 entry to explicit `cache_mgr_v1`/`cache_mgr_v2` variants, and removes Gemma3 from B300 post-merge list.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The PR description is still the template and lacks a real title, issue summary, and test coverage details.	Replace the template with a filled-in PR title, a short description of what changed and why, and concrete test coverage for the new paths.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and clearly summarizes the main change: adding Gemma and GLM disaggregated Python transceiver tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/defs/accuracy/references/gsm8k.yaml`:
- Around line 461-465: The GSM8K accuracy reference for zai-org/GLM-5 currently
uses a placeholder NVFP4 threshold copied from FP8, so update the NVFP4 baseline
in the gsm8k.yaml reference with the measured GLM-5 NVFP4 accuracy from the test
run before merging. Keep the entry under the zai-org/GLM-5 quant_algo: NVFP4
block, and ensure the value matches the enforced threshold used by the accuracy
check in accuracy_core.py so the pass/fail gate reflects the real NVFP4 result.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5de666c9-17b4-4e06-8d74-c730efc23d4e

📥 Commits

Reviewing files that changed from the base of the PR and between 92147d6 and a413c98.

📒 Files selected for processing (6)

tests/integration/defs/accuracy/references/gsm8k.yaml
tests/integration/defs/accuracy/test_disaggregated_serving.py
tests/integration/test_lists/qa/llm_function_core.txt
tests/integration/test_lists/test-db/l0_dgx_b200.yml
tests/integration/test_lists/test-db/l0_dgx_b300.yml
tests/integration/test_lists/test-db/l0_dgx_h100.yml

💤 Files with no reviewable changes (1)

tests/integration/test_lists/test-db/l0_dgx_b300.yml

chuangz0 · 2026-07-01T01:16:00Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-01T01:21:30Z

PR_Github #56776 [ run ] triggered by Bot. Commit: 824a082 Link to invocation

tensorrt-cicd · 2026-07-01T03:34:17Z

PR_Github #56776 [ run ] completed with state SUCCESS. Commit: 824a082
/LLM/main/L0_MergeRequest_PR pipeline #45592 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

chuangz0 · 2026-07-02T09:40:17Z

/bot run --disable-fail-fast

chuangz0 · 2026-07-03T01:30:15Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-03T01:36:33Z

PR_Github #57293 [ run ] triggered by Bot. Commit: f022818 Link to invocation

tensorrt-cicd · 2026-07-03T02:22:20Z

PR_Github #57293 [ run ] completed with state FAILURE. Commit: f022818
/LLM/main/L0_MergeRequest_PR pipeline #46053 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 · 2026-07-03T02:24:40Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-03T02:31:16Z

PR_Github #57316 [ run ] triggered by Bot. Commit: 8905f47 Link to invocation

tensorrt-cicd · 2026-07-03T08:38:34Z

PR_Github #57316 [ run ] completed with state FAILURE. Commit: 8905f47
/LLM/main/L0_MergeRequest_PR pipeline #46074 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 · 2026-07-06T02:52:01Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-06T02:57:32Z

PR_Github #57668 [ run ] triggered by Bot. Commit: 846b0ac Link to invocation

tensorrt-cicd · 2026-07-06T05:14:47Z

PR_Github #57668 [ run ] completed with state SUCCESS. Commit: 846b0ac
/LLM/main/L0_MergeRequest_PR pipeline #46388 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

chuangz0 · 2026-07-06T05:46:44Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-06T05:53:10Z

PR_Github #57698 [ run ] triggered by Bot. Commit: 878ca8f Link to invocation

tensorrt-cicd · 2026-07-06T08:16:26Z

PR_Github #57698 [ run ] completed with state SUCCESS. Commit: 878ca8f
/LLM/main/L0_MergeRequest_PR pipeline #46415 completed with status: 'SUCCESS'

CI Report

Link to invocation

…#15755) Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 requested review from a team as code owners June 30, 2026 04:29

github-actions Bot assigned chuangz0 Jun 30, 2026

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from a73e3f2 to a413c98 Compare June 30, 2026 04:29

coderabbitai Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread tests/integration/defs/accuracy/references/gsm8k.yaml Outdated

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from a413c98 to 824a082 Compare July 1, 2026 01:15

ruodil approved these changes Jul 1, 2026

View reviewed changes

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from 824a082 to f022818 Compare July 2, 2026 09:39

xinhe-nv approved these changes Jul 3, 2026

View reviewed changes

chuangz0 added 4 commits July 3, 2026 10:24

add gemma and glm disagg python transceiver test

2f7e5a3

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

update glm52 disagg test

ed8d51e

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

update glm52 disagg test

97a5704

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

fix disagg test

8905f47

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from f022818 to 8905f47 Compare July 3, 2026 02:24

fix line

878ca8f

Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from 8905f47 to 846b0ac Compare July 6, 2026 02:51

chuangz0 force-pushed the add_disagg_minimax_and_glm_acc branch from 846b0ac to 878ca8f Compare July 6, 2026 03:22

Shixiaowei02 approved these changes Jul 6, 2026

View reviewed changes

chuangz0 merged commit 9b593dd into NVIDIA:main Jul 7, 2026
7 checks passed

BrianLi23 pushed a commit to BrianLi23/TensorRT-LLM that referenced this pull request Jul 9, 2026

[None][feat] add gemma and glm disagg python transceiver test (NVIDIA…

c459c88

…#15755) Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com>

Uh oh!

Conversation

chuangz0 commented Jun 30, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

chuangz0 commented Jun 30, 2026

Uh oh!

coderabbitai Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (2 warnings)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chuangz0 commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

tensorrt-cicd commented Jul 1, 2026

Uh oh!

chuangz0 commented Jul 2, 2026

Uh oh!

chuangz0 commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

chuangz0 commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

tensorrt-cicd commented Jul 3, 2026

Uh oh!

chuangz0 commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

chuangz0 commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

tensorrt-cicd commented Jul 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

chuangz0 commented Jun 30, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading