Skip to content

[None][test] Waive 8 failed cases for main in QA CI#14309

Merged
crazydemo merged 1 commit into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260519-131254
May 20, 2026
Merged

[None][test] Waive 8 failed cases for main in QA CI#14309
crazydemo merged 1 commit into
NVIDIA:mainfrom
xinhe-nv:trtllm-ci-report/waive-20260519-131254

Conversation

@xinhe-nv
Copy link
Copy Markdown
Collaborator

@xinhe-nv xinhe-nv commented May 19, 2026

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 6184906, 6184914, 6190759

Waive entries added

test_e2e.py::test_openai_disagg_multi_nodes_completion[ctx_tp1pp2-gen_tp1pp2] SKIP (https://nvbugs/6190759)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales_4gpus[pp4-mtp_nextn=0-fp8kv=False-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=False-sampler_async_worker=False] SKIP (https://nvbugs/6184914)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-pp4-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/6184914)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=2-pp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/6184914)
disaggregated/test_disaggregated.py::test_disaggregated_cuda_graph[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6184906)
disaggregated/test_disaggregated.py::test_disaggregated_conditional[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6184906)
disaggregated/test_disaggregated.py::test_disaggregated_single_gpu[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6184906)
disaggregated/test_disaggregated.py::test_disaggregated_kv_cache_time_output[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6184906)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Updated integration test skip list with adjustments to existing entries and new test variants.

Review Change Stack

Bug(s): 6184906, 6184914, 6190759
Requested by: qa@nvidia.com

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
@crazydemo crazydemo changed the title [None][infra] Waive 8 failed cases for main in post-merge [None][test] Waive 8 failed cases for main in QA CI May 19, 2026
@crazydemo crazydemo enabled auto-merge (squash) May 19, 2026 13:13
@crazydemo
Copy link
Copy Markdown
Collaborator

/bot run --stage-list ""

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 19, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 22e6bfb6-3c21-401f-a037-02e228b445e0

📥 Commits

Reviewing files that changed from the base of the PR and between 22bafe4 and fa0161d.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

Updates tests/integration/test_lists/waives.txt to skip additional integration test variants: DeepSeekV3Lite accuracy tests for fp8 block scales and fp4 quantization, disaggregated inference test scenarios for conditional execution and CUDA graphs, and an end-to-end disaggregated multi-node completion test.

Changes

Test Skip Waivers Update

Layer / File(s) Summary
DeepSeekV3Lite accuracy test waivers
tests/integration/test_lists/waives.txt
Adds new skip entry for TestDeepSeekV3Lite::test_fp8_block_scales_4gpus and replaces/updates TestDeepSeekV3Lite::test_nvfp4_4gpus waiver entries with additional parameter variants and updated bug references.
Disaggregated inference test waivers
tests/integration/test_lists/waives.txt
Expands disaggregated test waiver blocks by adding test_disaggregated_conditional, test_disaggregated_cuda_graph, test_disaggregated_kv_cache_time_output, and test_disaggregated_single_gpu entries while re-listing existing ctxpp/ctxtp and perf-metrics parameterized waivers.
End-to-end disaggregated test waiver
tests/integration/test_lists/waives.txt
Adds new skip entry for test_e2e.py::test_openai_disagg_multi_nodes_completion with specified parameter configuration and bug reference.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#13986: Both PRs update test waivers for TestDeepSeekV3Lite::test_nvfp4_4gpus and test_fp8_block_scales_4gpus in the same waiver list file with overlapping test entries.
  • NVIDIA/TensorRT-LLM#14163: Both PRs modify tests/integration/test_lists/waives.txt to add and adjust integration-test SKIP entries across different test suites.
  • NVIDIA/TensorRT-LLM#13204: Both PRs update tests/integration/test_lists/waives.txt by modifying SKIP entries for disaggregated/test_disaggregated.py with ctxpp-related variants.

Suggested reviewers

  • jieli-matrix
  • StanleySun639
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title follows the required template format [None][type] Summary and clearly indicates this is a waive PR for 8 failed test cases, matching the PR's primary purpose.
Description check ✅ Passed The description is auto-generated and comprehensive, including bug references, specific waive entries, and all 8 test cases being skipped, though it doesn't follow the template's optional sections like Test Coverage and PR Checklist.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #49196 [ run ] triggered by Bot. Commit: fa0161d Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #49196 [ run ] completed with state SUCCESS. Commit: fa0161d
/LLM/main/L0_MergeRequest_PR pipeline #38872 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@crazydemo
Copy link
Copy Markdown
Collaborator

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #49311 [ reuse-pipeline ] triggered by Bot. Commit: fa0161d Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #49311 [ reuse-pipeline ] completed with state SUCCESS. Commit: fa0161d
Reusing PR_Github #49196 (Partly Tested) for commit fa0161d

Link to invocation

@crazydemo crazydemo merged commit 6de1482 into NVIDIA:main May 20, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants