Skip to content

[https://nvbugs/5821415][test] update rtx6k test list#14929

Merged
xinhe-nv merged 3 commits into
NVIDIA:mainfrom
xinhe-nv:update-rtx6k
Jun 4, 2026
Merged

[https://nvbugs/5821415][test] update rtx6k test list#14929
xinhe-nv merged 3 commits into
NVIDIA:mainfrom
xinhe-nv:update-rtx6k

Conversation

@xinhe-nv
Copy link
Copy Markdown
Collaborator

@xinhe-nv xinhe-nv commented Jun 4, 2026

@coderabbitai summary

Description

  • remove all llama tests from test list.
  • sort test list, remove dup lines.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@xinhe-nv xinhe-nv requested a review from litaotju June 4, 2026 02:48
@xinhe-nv xinhe-nv requested a review from a team as a code owner June 4, 2026 02:48
@xinhe-nv xinhe-nv enabled auto-merge (squash) June 4, 2026 02:50
@xinhe-nv
Copy link
Copy Markdown
Collaborator Author

xinhe-nv commented Jun 4, 2026

/bot run --stage-list ""

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 4, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR updates the test selection list for RTX6k QA environments. It expands the DeepSeekV3Lite bfloat16 parameter matrices with new cuda_graph and attention_dp configurations, adjusts existing parameter combinations for different mtp_nextn values, and modifies the selection of downstream model variant tests (nvfp4, guided decoding) and cross-suite accuracy/e2e test entries.

Changes

RTX6k Test Matrix Selection

Layer / File(s) Summary
DeepSeekV3Lite bfloat16 parameter matrices
tests/integration/test_lists/qa/llm_function_rtx6k.txt
Expands mtp_nextn=0 matrix with cuda_graph=True and attention_dp=True configurations (with overlap_scheduler=False across torch_compile variants). Adjusts mtp_nextn=2 parameter combinations for cuda_graph, overlap_scheduler, torch_compile, and v2_kv_cache.
DeepSeekV3Lite variant and downstream test selection
tests/integration/test_lists/qa/llm_function_rtx6k.txt
Updates enabled DeepSeekV3Lite variant test identifiers (bfloat16_4gpus_online_eplb, guided decoding variants, nvfp4, nvfp4_4gpus_online_eplb, batch-waiting scenarios) with entries newly added, removed, or reordered.
Cross-model accuracy and e2e test selection
tests/integration/test_lists/qa/llm_function_rtx6k.txt
Modifies selection and order of test entries across GPTOSS, model accuracy tests, encode/multimodal, e2e, and disaggregated test suites.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#14866: Both PRs adjust handling of TestDeepSeekV3Lite::test_bfloat16 parameterizations involving mtp_nextn and torch_compile (main PR via test list selection, related PR via test skip list).

Suggested reviewers

  • StanleySun639
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately describes the main change: updating the RTX6K test list file (llm_function_rtx6k.txt) by removing llama tests and reorganizing entries.
Description check ✅ Passed The description provides specific details about the changes: removing all llama tests and sorting/deduplicating the test list, though the Test Coverage section remains unfilled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/integration/test_lists/qa/llm_function_rtx6k.txt (1)

22-33: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Missing attention_dp=True + cuda_graph=True + overlap_scheduler=False coverage for mtp_nextn=0.

Coverage looks incomplete for the intended DeepSeekV3Lite bfloat16 matrix expansion. Please add these entries to tests/integration/test_lists/qa/llm_function_rtx6k.txt:

  • accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=False]
  • accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True]
  • accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=False]
  • accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=True]

Coverage status: insufficient for this matrix slice in this PR cohort.

As per coding guidelines, for files under tests/** feedback should be actionable and explicitly state whether coverage is sufficient or insufficient.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/qa/llm_function_rtx6k.txt` around lines 22 - 33,
The test list is missing four parameterized cases for TestDeepSeekV3Lite
bfloat16 with mtp_nextn=0 and attention_dp=True + cuda_graph=True +
overlap_scheduler=False; edit
tests/integration/test_lists/qa/llm_function_rtx6k.txt and add the four entries
for accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16 with the
combinations torch_compile={False,True} and v2_kv_cache={False,True} (as listed
in the review comment) so the matrix slice
(attention_dp=True,cuda_graph=True,overlap_scheduler=False,mtp_nextn=0) is fully
covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tests/integration/test_lists/qa/llm_function_rtx6k.txt`:
- Around line 22-33: The test list is missing four parameterized cases for
TestDeepSeekV3Lite bfloat16 with mtp_nextn=0 and attention_dp=True +
cuda_graph=True + overlap_scheduler=False; edit
tests/integration/test_lists/qa/llm_function_rtx6k.txt and add the four entries
for accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16 with the
combinations torch_compile={False,True} and v2_kv_cache={False,True} (as listed
in the review comment) so the matrix slice
(attention_dp=True,cuda_graph=True,overlap_scheduler=False,mtp_nextn=0) is fully
covered.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6636a6ec-60ff-4044-a23c-b8a9042c2bc5

📥 Commits

Reviewing files that changed from the base of the PR and between e1212ad and 5d3fbcb.

📒 Files selected for processing (1)
  • tests/integration/test_lists/qa/llm_function_rtx6k.txt

@xinhe-nv
Copy link
Copy Markdown
Collaborator Author

xinhe-nv commented Jun 4, 2026

/bot run --stage-list ""

@xinhe-nv xinhe-nv changed the title [None][test] update rtx6k test list [https://nvbugs/5821415][test] update rtx6k test list Jun 4, 2026
@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51946 [ run ] triggered by Bot. Commit: 06941d7 Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51948 [ run ] triggered by Bot. Commit: 06941d7 Link to invocation

@NVIDIA NVIDIA deleted a comment from tensorrt-cicd Jun 4, 2026
@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51948 [ run ] completed with state SUCCESS. Commit: 06941d7
/LLM/main/L0_MergeRequest_PR pipeline #41298 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

xinhe-nv added 3 commits June 4, 2026 12:18
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv
Copy link
Copy Markdown
Collaborator Author

xinhe-nv commented Jun 4, 2026

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51969 [ reuse-pipeline ] triggered by Bot. Commit: 64d5740 Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51969 [ reuse-pipeline ] completed with state SUCCESS. Commit: 64d5740
Reusing PR_Github #51948 (Partly Tested) for commit 64d5740

Link to invocation

@xinhe-nv xinhe-nv merged commit b7ca2a6 into NVIDIA:main Jun 4, 2026
7 checks passed
@xinhe-nv xinhe-nv deleted the update-rtx6k branch June 4, 2026 05:16
yufeiwu-nv pushed a commit to yufeiwu-nv/TensorRT-LLM that referenced this pull request Jun 4, 2026
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants