Skip to content

[TPU] Update torch-xla version to include paged attention tuned block change #19813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 18, 2025

Conversation

QiliangCui
Copy link
Contributor

@QiliangCui QiliangCui commented Jun 18, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Update torch-xla version to include paged attention tuned block change

Test Plan

Run vLLM benchmark with new package and compare with old

comparing with hourly run.

Test Result

MODEL Device 20250618_100001 with new package
meta-llama/Llama-3.1-70B-Instruct v6e-8 5.48 5.43
meta-llama/Llama-3.1-8B-Instruct v6e-1 7.81 7.82
meta-llama/Llama-3.3-70B-Instruct v6e-8 5.42 5.48

(Optional) Documentation Update

N/A

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
@QiliangCui QiliangCui marked this pull request as draft June 18, 2025 16:27
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @QiliangCui, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on upgrading the core PyTorch and PyTorch/XLA dependencies for TPU environments to a more recent nightly build. The primary objective is to integrate a specific performance optimization related to paged attention, aiming to enhance the efficiency of model execution on TPUs.

Highlights

  • Dependency Update: I've updated the versions of torch, torchvision, and torch_xla in requirements/tpu.txt to a newer nightly build, specifically from 2.8.0.dev20250617 to 2.8.0.dev20250618.
  • Performance Improvement: This update is intended to incorporate a specific 'paged attention tuned block change' within torch-xla, which should lead to performance improvements on TPU environments.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the torch, torchvision, and torch_xla dependencies to version 2.8.0.dev20250618 for the TPU requirements. The change is minimal, focusing solely on the version bump. It is important to verify the impact of this update, as noted in the specific comment regarding the test plan and results.

@mergify mergify bot added ci/build qwen Related to Qwen models labels Jun 18, 2025
@QiliangCui QiliangCui marked this pull request as ready for review June 18, 2025 18:16
@yaochengji yaochengji added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 18, 2025
Copy link
Collaborator

@yaochengji yaochengji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@yaochengji yaochengji enabled auto-merge (squash) June 18, 2025 21:55
@yaochengji yaochengji merged commit 04fefe7 into vllm-project:main Jun 18, 2025
60 checks passed
yeqcharlotte pushed a commit to yeqcharlotte/vllm that referenced this pull request Jun 22, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
minpeter pushed a commit to minpeter/vllm that referenced this pull request Jun 24, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Signed-off-by: minpeter <kali2005611@gmail.com>
yangw-dev pushed a commit to yangw-dev/vllm that referenced this pull request Jun 24, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Signed-off-by: Yang Wang <elainewy@meta.com>
gmarinho2 pushed a commit to gmarinho2/vllm that referenced this pull request Jun 26, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
xjpang pushed a commit to xjpang/vllm that referenced this pull request Jun 30, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
wseaton pushed a commit to wseaton/vllm that referenced this pull request Jun 30, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
wseaton pushed a commit to wseaton/vllm that referenced this pull request Jun 30, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
wwl2755-google pushed a commit to wwl2755-google/vllm that referenced this pull request Jul 1, 2025
… change (vllm-project#19813)

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/build qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants