[https://nvbugs/5821415][test] update rtx6k test list#14929
Conversation
|
/bot run --stage-list "" |
📝 WalkthroughWalkthroughThis PR updates the test selection list for RTX6k QA environments. It expands the DeepSeekV3Lite bfloat16 parameter matrices with new cuda_graph and attention_dp configurations, adjusts existing parameter combinations for different mtp_nextn values, and modifies the selection of downstream model variant tests (nvfp4, guided decoding) and cross-suite accuracy/e2e test entries. ChangesRTX6k Test Matrix Selection
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/integration/test_lists/qa/llm_function_rtx6k.txt (1)
22-33:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMissing
attention_dp=True + cuda_graph=True + overlap_scheduler=Falsecoverage formtp_nextn=0.Coverage looks incomplete for the intended DeepSeekV3Lite bfloat16 matrix expansion. Please add these entries to
tests/integration/test_lists/qa/llm_function_rtx6k.txt:
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=False]accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True]accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=False]accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=0-attention_dp=True-cuda_graph=True-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=True]Coverage status: insufficient for this matrix slice in this PR cohort.
As per coding guidelines, for files under
tests/**feedback should be actionable and explicitly state whether coverage is sufficient or insufficient.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/test_lists/qa/llm_function_rtx6k.txt` around lines 22 - 33, The test list is missing four parameterized cases for TestDeepSeekV3Lite bfloat16 with mtp_nextn=0 and attention_dp=True + cuda_graph=True + overlap_scheduler=False; edit tests/integration/test_lists/qa/llm_function_rtx6k.txt and add the four entries for accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16 with the combinations torch_compile={False,True} and v2_kv_cache={False,True} (as listed in the review comment) so the matrix slice (attention_dp=True,cuda_graph=True,overlap_scheduler=False,mtp_nextn=0) is fully covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@tests/integration/test_lists/qa/llm_function_rtx6k.txt`:
- Around line 22-33: The test list is missing four parameterized cases for
TestDeepSeekV3Lite bfloat16 with mtp_nextn=0 and attention_dp=True +
cuda_graph=True + overlap_scheduler=False; edit
tests/integration/test_lists/qa/llm_function_rtx6k.txt and add the four entries
for accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16 with the
combinations torch_compile={False,True} and v2_kv_cache={False,True} (as listed
in the review comment) so the matrix slice
(attention_dp=True,cuda_graph=True,overlap_scheduler=False,mtp_nextn=0) is fully
covered.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 6636a6ec-60ff-4044-a23c-b8a9042c2bc5
📒 Files selected for processing (1)
tests/integration/test_lists/qa/llm_function_rtx6k.txt
|
/bot run --stage-list "" |
|
PR_Github #51946 [ run ] triggered by Bot. Commit: |
|
PR_Github #51948 [ run ] triggered by Bot. Commit: |
|
PR_Github #51948 [ run ] completed with state |
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
|
/bot reuse-pipeline |
|
PR_Github #51969 [ reuse-pipeline ] triggered by Bot. Commit: |
|
PR_Github #51969 [ reuse-pipeline ] completed with state |
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com> Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
@coderabbitai summary
Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.