Skip to content

[https://nvbugs/6168136][fix] Unwaive GPT-OSS test_w4_4gpus dp4-trtllm-fp8#14118

Merged
xwang233 merged 1 commit into
NVIDIA:mainfrom
xwang233:fix/unwaive-6168136
May 15, 2026
Merged

[https://nvbugs/6168136][fix] Unwaive GPT-OSS test_w4_4gpus dp4-trtllm-fp8#14118
xwang233 merged 1 commit into
NVIDIA:mainfrom
xwang233:fix/unwaive-6168136

Conversation

@xwang233
Copy link
Copy Markdown
Collaborator

@xwang233 xwang233 commented May 14, 2026 •
edited by coderabbitai Bot
Loading

Summary

  • Removes the SKIP for TestGPTOSS::test_w4_4gpus[v1_kv_cache-dp4-trtllm-fp8] from tests/integration/test_lists/waives.txt.
  • NVBug 6168136 was auto-filed against this test but offline reproduction on 4× GB200 did not reproduce a failure. NVBug will be set to Can't Reproduce.

Test plan

Trigger pre-merge CI with /bot run --stage-list "GB200-4_GPUs-PyTorch-2" to exercise the un-waived test.

Summary by CodeRabbit

  • Tests
    • Updated test waiver list by removing an outdated entry.

Review Change Stack

@xwang233
[https://nvbugs/6168136][fix] Unwaive TestGPTOSS::test_w4_4gpus[v1_kv…
1aa55e7 
…_cache-dp4-trtllm-fp8]

The test was waived after a single round-4 pytest-timeout in pre-merge
build #37652 (PR NVIDIA#13910). Investigation shows that's a false-positive
auto-file:

- Same build's rounds 1-3 all PASSED this test (14:43 / 16:21 / 19:01)
  but their results.xml were dropped at SLURM TimeLimit with the
  Jenkins log line 'Stage is interrupted, skip to upload test result.'
  Only round 4's results.xml reached the bot, and on round 4 the test
  happened to hit pytest's per-test --timeout=3600 instead of finishing
  in the usual ~10-20 min.
- Reproduced on dlcluster 4x GB200: 11/11 PASS (job 1003130: 1 PASS at
  13:29; job 1003278: 10-iter loop, all PASS, 8:42-16:53, mean 10:44).
- PR NVIDIA#13910 is a 2-file gc-threshold refactor with no GPT-OSS / MoE /
  KV-cache / test-infra code touched, so it cannot be the cause; it
  was approved and merged to main ~10h after this NVBug was filed.

Across all known executions of this exact parameterization on real
GB200, 14/15 are PASS; the one cliff event was on nvl72129-T18, a node
not reachable from the lease used for the repro.

Signed-off-by: Xiao Wang <24860335+xwang233@users.noreply.github.com>
@xwang233 xwang233 force-pushed the fix/unwaive-6168136 branch from 5e752d1 to 1aa55e7 Compare May 14, 2026 05:38
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 14, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 617317d1-a524-4e1d-85f6-73fe5397bfae

📥 Commits

Reviewing files that changed from the base of the PR and between 27fdd55 and 1aa55e7.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt
📝 Walkthrough

Walkthrough

This PR removes a single SKIP waiver entry from the integration test waiver list for the TestGPTOSS::test_w4_4gpus[v1_kv_cache-dp4-trtllm-fp8] test case, enabling the test to run again after the underlying issue (nvbugs/6168136) is resolved.

Changes

Test Waiver Update

Layer / File(s) Summary
Remove test_w4_4gpus SKIP waiver
tests/integration/test_lists/waives.txt		 The SKIP waiver for accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-dp4-trtllm-fp8] is removed from the waiver list, allowing the test to execute.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#14058: Both PRs modify tests/integration/test_lists/waives.txt to remove SKIP waivers for specific LLM API test cases.
  • NVIDIA/TensorRT-LLM#14114: Both PRs modify tests/integration/test_lists/waives.txt by removing SKIP waiver entries.
  • NVIDIA/TensorRT-LLM#14086: Both PRs modify tests/integration/test_lists/waives.txt by adding/removing SKIP waiver entries for accuracy/test_llm_api_pytorch.py test cases.

Suggested reviewers

  • jieli-matrix
  • LarryXFly
  • xinhe-nv
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: removing a test waiver (unwaiving) for a specific GPT-OSS test case related to NVBug 6168136.
Description check ✅ Passed The description includes the required PR title format with NVBug ID and type, provides a clear summary of changes, and includes a test plan. It does not follow the template format exactly but addresses the key requirements.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

Comment @coderabbitai help to get the list of available commands and usage tips.

@xwang233
Copy link
Copy Markdown
Collaborator Author

xwang233 commented May 14, 2026

/bot run --stage-list "GB200-4_GPUs-PyTorch-2"

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented May 14, 2026

PR_Github #48303 [ run ] triggered by Bot. Commit: 1aa55e7 Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented May 14, 2026

PR_Github #48303 [ run ] completed with state SUCCESS. Commit: 1aa55e7
/LLM/main/L0_MergeRequest_PR pipeline #38114 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@xwang233 xwang233 enabled auto-merge (squash) May 15, 2026 04:06
@xwang233
Copy link
Copy Markdown
Collaborator Author

xwang233 commented May 15, 2026

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented May 15, 2026

PR_Github #48521 [ reuse-pipeline ] triggered by Bot. Commit: 1aa55e7 Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented May 15, 2026

PR_Github #48521 [ reuse-pipeline ] completed with state SUCCESS. Commit: 1aa55e7
Reusing PR_Github #48303 (Partly Tested) for commit 1aa55e7

Link to invocation

@xwang233 xwang233 merged commit b0b0052 into NVIDIA:main May 15, 2026
13 of 14 checks passed
This was referenced May 15, 2026
Open
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@niukuo niukuo niukuo approved these changes

@ZhanruiSunCh ZhanruiSunCh Awaiting requested review from ZhanruiSunCh

@zhenhuaw-me zhenhuaw-me Awaiting requested review from zhenhuaw-me

Assignees

@xwang233 xwang233

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@xwang233 @tensorrt-cicd @niukuo