Skip to content

[None][chore] Add failed cases into waives.txt#12363

Merged
xinhe-nv merged 2 commits intoNVIDIA:mainfrom
xinhe-nv:user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012
Mar 20, 2026
Merged

[None][chore] Add failed cases into waives.txt#12363
xinhe-nv merged 2 commits intoNVIDIA:mainfrom
xinhe-nv:user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012

Conversation

@xinhe-nv
Copy link
Collaborator

@xinhe-nv xinhe-nv commented Mar 19, 2026

waive failed cases.

Summary by CodeRabbit

  • Tests
    • Removed multimodal 2-GPU quickstart test cases and associated test duration metrics.
    • Cleaned up QA test list by removing related test entries.
    • Test infrastructure maintenance with no impact to end-user functionality.

@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012 branch 2 times, most recently from 55e223d to cbf9e84 Compare March 20, 2026 02:08
@xinhe-nv xinhe-nv marked this pull request as ready for review March 20, 2026 02:08
@xinhe-nv xinhe-nv requested review from a team as code owners March 20, 2026 02:08
@xinhe-nv xinhe-nv enabled auto-merge (squash) March 20, 2026 02:08
@xinhe-nv
Copy link
Collaborator Author

/bot run --stage-list ""

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 20, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bc34c97d-ffe1-4256-9784-53cb3c1e9797

📥 Commits

Reviewing files that changed from the base of the PR and between 297fa20 and cbf9e84.

📒 Files selected for processing (3)
  • tests/integration/defs/.test_durations
  • tests/integration/defs/test_e2e.py
  • tests/integration/test_lists/qa/llm_function_core.txt
💤 Files with no reviewable changes (3)
  • tests/integration/test_lists/qa/llm_function_core.txt
  • tests/integration/defs/test_e2e.py
  • tests/integration/defs/.test_durations

📝 Walkthrough

Walkthrough

This PR removes the test_ptp_quickstart_multimodal_2gpu test case from the E2E test suite, including its associated duration mappings and QA list references. The test was used for validating multimodal models with 2-GPU tensor parallelism.

Changes

Cohort / File(s) Summary
E2E Test Removal
tests/integration/defs/test_e2e.py
Removed the test_ptp_quickstart_multimodal_2gpu function entirely, including 2-GPU skip markers, parametrization for Mistral-Small-3.1-24B-Instruct, multimodal image prompts, tensor parallelism configuration, and accuracy assertions.
Test Duration & QA List Cleanup
tests/integration/defs/.test_durations, tests/integration/test_lists/qa/llm_function_core.txt
Removed three duration mappings for the deleted multimodal 2-GPU test across model variants, and removed corresponding E2E QA entry and a separate tokenizer multinode test from the QA execution list.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

  • jieli-matrix
  • StanleySun639
  • LarryXFly
  • crazydemo
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description is extremely brief and vague. While the template requires a Description section with explanation of the issue and solution, the provided description only states 'waive failed cases' without context. Expand the description to explain which specific test cases are being waived and why (e.g., known failures, infrastructure issues, pending fixes).
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding failed test cases to waives.txt.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

You can disable the changed files summary in the walkthrough.

Disable the reviews.changed_files_summary setting to disable the changed files summary in the walkthrough.

@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012 branch from cbf9e84 to 178df6a Compare March 20, 2026 02:14
@tensorrt-cicd
Copy link
Collaborator

PR_Github #39666 [ run ] triggered by Bot. Commit: 178df6a Link to invocation

@tensorrt-cicd
Copy link
Collaborator

PR_Github #39666 [ run ] completed with state FAILURE. Commit: 178df6a

Link to invocation

@xinhe-nv
Copy link
Collaborator Author

/bot run --stage-list ""

@tensorrt-cicd
Copy link
Collaborator

PR_Github #39671 [ run ] triggered by Bot. Commit: 178df6a Link to invocation

@tensorrt-cicd
Copy link
Collaborator

PR_Github #39671 [ run ] completed with state SUCCESS. Commit: 178df6a
/LLM/main/L0_MergeRequest_PR pipeline #30873 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>
@xinhe-nv xinhe-nv force-pushed the user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012 branch from 4cd9f96 to ff085ff Compare March 20, 2026 05:49
@xinhe-nv
Copy link
Collaborator Author

/bot resue-pipeline

@github-actions
Copy link

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental) --high-priority]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

--high-priority (OPTIONAL) : Run the pipeline with high priority. This option is restricted to authorized users only and will route the job to a high-priority queue.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@xinhe-nv
Copy link
Collaborator Author

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #39700 [ reuse-pipeline ] triggered by Bot. Commit: ff085ff Link to invocation

@tensorrt-cicd
Copy link
Collaborator

PR_Github #39700 [ reuse-pipeline ] completed with state SUCCESS. Commit: ff085ff
Reusing PR_Github #39671 (Partly Tested) for commit ff085ff

Link to invocation

@xinhe-nv xinhe-nv merged commit c74f0d6 into NVIDIA:main Mar 20, 2026
5 checks passed
@xinhe-nv xinhe-nv deleted the user/qa/post_update_waive_20260320_LLM_FUNCTION_TEST_2012 branch March 20, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants