Skip to content

Comments

Add stack for qkv projection#1848

Closed
drisspg wants to merge 1 commit intomainfrom
vllm-no-change
Closed

Add stack for qkv projection#1848
drisspg wants to merge 1 commit intomainfrom
vllm-no-change

Conversation

@drisspg
Copy link
Contributor

@drisspg drisspg commented Mar 7, 2025

No description provided.

@pytorch-bot
Copy link

pytorch-bot bot commented Mar 7, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1848

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 2aa9130 with merge base ffb4350 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 7, 2025
# Handle scales based on their dimensionality
if all(t.scale.ndim == 0 for t in tensors):
# Use the maximum scale value from all tensors for numerical stability
stacked_scale = torch.max(torch.cat([t.scale for t in tensors]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this change numerics if max_scale != original scale? I would have expected something like a rescale of the data if the scale is changed.

@drisspg drisspg closed this Jul 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants