Skip to content

[None][test] Waive 8 failed cases for main in QA CI#16030

Merged
jieli-matrix merged 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260707-034028
Jul 7, 2026
Merged

[None][test] Waive 8 failed cases for main in QA CI#16030
jieli-matrix merged 1 commit into
NVIDIA:mainfrom
trtllm-agent:trtllm-ci-report/waive-20260707-034028

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Bug(s): 6414760, 6422318, 6422334, 6422343, 6422351

Waive entries added

full:H20/accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[nvidia_Llama-3.1-8B-Instruct-FP8-True] SKIP (https://nvbugs/6422351)
full:H20/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=True-enable_chunked_prefill=True-v2_kv_cache=True] SKIP (https://nvbugs/6422343)
full:H20/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=True-enable_chunked_prefill=True-v2_kv_cache=False] SKIP (https://nvbugs/6422343)
full:H20/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[llguidance-eagle3_one_model=False] SKIP (https://nvbugs/6422334)
full:H20/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[xgrammar-eagle3_one_model=False] SKIP (https://nvbugs/6422334)
full:H20/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[full_budget] SKIP (https://nvbugs/6422318)
full:H20/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6422318)
full:H20/test_e2e.py::test_qwen_e2e_cpprunner_large_new_tokens[DeepSeek-R1-Distill-Qwen-1.5B-DeepSeek-R1-Distill-Qwen-1.5B] SKIP (https://nvbugs/6414760)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

  • Tests
    • Updated the list of waived integration tests for H20-related runs.
    • Added waivers for several model and configuration combinations, including guided decoding, multimodal, and end-to-end runner cases.
    • Reorganized existing waiver entries to reflect the latest test coverage.

Bug(s): 6414760, 6422318, 6422334, 6422343, 6422351
Requested by: qa@nvidia.com

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
@jieli-matrix jieli-matrix enabled auto-merge (squash) July 7, 2026 03:40
@jieli-matrix

Copy link
Copy Markdown
Collaborator

/bot skip --comment "Only waive tests"

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 57733538-2704-4999-90b8-87a305ede546

📥 Commits

Reviewing files that changed from the base of the PR and between 790ea15 and 33affc9.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

This PR updates the waives.txt integration test list, modifying the H20-scoped waiver section by adding new SKIP entries for Llama3_1_8BInstruct guided decoding, model registry accuracy, DeepSeekV3Lite bfloat16 variants, Exaone4_5_33B multimodal cases, and Qwen e2e tests, while retaining prior GPTOSS and DeepSeek waivers.

Changes

Waive List Update

Layer / File(s) Summary
H20 waiver entries
tests/integration/test_lists/waives.txt
Adds SKIP entries for Llama3_1_8BInstruct guided decoding (Eagle3, two variants), TestModelRegistryAccuracy autodeploy case, DeepSeekV3Lite bfloat16 mtp_nextn=2 variants (two), Exaone4_5_33B multimodal auto-dtype cases (two), and a Qwen e2e cpp runner case, while retaining prior GPTOSS eagle3_2gpus/w4_chunked_prefill and DeepSeek helix/overlap waives within the relocated block.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#15733: Both PRs add/replace SKIP waiver entries for DeepSeekV3Lite guided decoding and bfloat16 variants in waives.txt.
  • NVIDIA/TensorRT-LLM#15902: Both PRs modify DeepSeekV3Lite helix-related waiver entries (test_auto_dtype_with_helix) in waives.txt.
  • NVIDIA/TensorRT-LLM#15977: Both PRs adjust H20 DeepSeek V3 Lite disaggregated/helix waive entries in the same file.

Suggested reviewers: crazydemo, LarryXFly, StanleySun639, xinhe-nv, jieli-matrix

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the waive PR for failed QA CI cases on main.
Description check ✅ Passed The description explains the auto-generated waive PR, includes related bugs, and lists the 8 waived cases.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57922 [ skip ] triggered by Bot. Commit: 33affc9 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57922 [ skip ] completed with state SUCCESS. Commit: 33affc9
Skipping testing for commit 33affc9

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants