Skip to content

Conversation

@zhxchen17
Copy link
Contributor

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Fixes #ISSUE_NUMBER

@zhxchen17 zhxchen17 requested a review from a team as a code owner October 29, 2025 02:37
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 29, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166494

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ You can merge normally! (1 Unrelated Failure)

As of commit 55c770c with merge base 4710fd9 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@huydhn
Copy link
Contributor

huydhn commented Oct 29, 2025

Thank you for the fix!

@zou3519
Copy link
Contributor

zou3519 commented Oct 29, 2025

Looks like we got problems. the vllm_basic_correctness_test appears to be a compile failure. Given it succeeds in vLLM CI I suspect a compile issue

@desertfire
Copy link
Contributor

Should we make the pin update automatic?

@huydhn
Copy link
Contributor

huydhn commented Oct 29, 2025

Should we make the pin update automatic?

It's automated, e.g. #165274, but that PR has been blocked for 2 weeks now. So, we end up with this manual update to coordinate the fix from vLLM and PyTorch

@zhxchen17
Copy link
Contributor Author

zhxchen17 commented Oct 30, 2025

Update: I tried to repro the test failures locally and seems 80% of tests (e.g. lora ones) stuck on my local and never finish (maybe I setup something wrong).

For the remaining 20% tests I can repro the failures, and it's still there when I turn off VLLM_USE_AOT_COMPILE, indicating it's more likely general compiler/vllm issues on the trunk. I can keep an eye on this but maybe we should get someone else to take a look for now.

@zhxchen17 zhxchen17 force-pushed the zhxchen17/ci/vllm_pin branch 3 times, most recently from 42a2703 to 83740c5 Compare November 24, 2025 18:40
@zhxchen17 zhxchen17 force-pushed the zhxchen17/ci/vllm_pin branch 5 times, most recently from a96c412 to a5916ab Compare November 30, 2025 05:05
@huydhn
Copy link
Contributor

huydhn commented Dec 1, 2025

I have temporarily marked vLLM trunk jobs as unstable in #169298 because we now need this change from vLLM vllm-project/vllm#29588 to fix an upstream issue from HuggingFace. Let's aim to update the pinned commit to latest vLLM main in the next couple of days

@zhxchen17 zhxchen17 force-pushed the zhxchen17/ci/vllm_pin branch from a5916ab to 893aad5 Compare December 5, 2025 14:50
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@zhxchen17 zhxchen17 force-pushed the zhxchen17/ci/vllm_pin branch from 893aad5 to 55c770c Compare December 5, 2025 17:44
@huydhn
Copy link
Contributor

huydhn commented Dec 5, 2025

@pytorchbot merge -f 'All vLLM tests are passing now, no need to run trunk jobs'

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

umechand-amd pushed a commit to ROCm/pytorch that referenced this pull request Dec 8, 2025
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Fixes #ISSUE_NUMBER

Pull Request resolved: pytorch#166494
Approved by: https://github.com/huydhn
JacobSzwejbka pushed a commit that referenced this pull request Dec 8, 2025
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Fixes #ISSUE_NUMBER

Pull Request resolved: #166494
Approved by: https://github.com/huydhn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants