Skip to content

[MPS] Speedup addmm #116548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 4 commits into from
Closed

[MPS] Speedup addmm #116548

wants to merge 4 commits into from

Conversation

malfet
Copy link
Contributor

@malfet malfet commented Dec 29, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

  • Do not copy bias to output
  • Skip respective multiplication op if either alpha or beta are equal to 1.0

@malfet
[MPS] Speedup addmm
de96e25 
- Do not copy bias to output
- Skip respective multiplication op if either alpha or beta are equal to 1.0

[ghstack-poisoned]
@malfet malfet requested a review from kulinseth as a code owner December 29, 2023 23:26
@malfet malfet mentioned this pull request Dec 29, 2023
Closed
@pytorch-bot pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels Dec 29, 2023
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Dec 29, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/116548

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a6a34e5 with merge base 97891b1 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@malfet
Update on "[MPS] Speedup addmm"
5b4805b 
- Do not copy bias to output
- Skip respective multiplication op if either alpha or beta are equal to 1.0

[ghstack-poisoned]
@malfet malfet requested review from Skylion007 and albanD December 29, 2023 23:28
@malfet malfet added the topic: improvements topic category label Dec 29, 2023
albanD
albanD reviewed Dec 30, 2023
View reviewed changes
aten/src/ATen/native/mps/operations/LinearAlgebra.mm
if (&output != &self) {
output.resize_(bias_sizes);
if (beta.toComplexDouble() != 0.0) {
output.copy_(*bias_);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is an out variant, overriding the output completely is not ok. We should add into it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and it is(all torch.addmm OpInfo tests use alpha and beta not equal to 1), this is why I'm removing this one, as copy is redundant.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I guess the next steps here would be to add this case to OpInfo and fix this kernel to have the appropriate behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think anything is needed here: addmm_out_mps copied bias to output for some reason, even though it wasn't at all needed, as MPSGraph always overwrites the output;

pytorch/aten/src/ATen/native/mps/operations/LinearAlgebra.mm

Lines 302 to 305 in 035e558

MPSGraphTensor* outputTensor = productTimesAlphaTensor;
if (is_beta_non_zero) {
outputTensor = [mpsGraph additionWithPrimaryTensor:productTimesAlphaTensor
secondaryTensor:biasTimesBetaTensor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, or perhaps I did not understand your original question: this override only happens if output != self, though yes, I'm not sure this kernel will work as expected if output = self, but imo this should be done as separate PR, this just eliminates unneeded multiplications if alpha and beta are 1 and unneeded override as function always writes to output

@malfet
Update on "[MPS] Speedup addmm"
eda0b06 
- Do not copy bias to output
- Skip respective multiplication op if either alpha or beta are equal to 1.0

[ghstack-poisoned]
@malfet
Update on "[MPS] Speedup addmm"
a6a34e5 
- Do not copy bias to output
- Skip respective multiplication op if either alpha or beta are equal to 1.0

[ghstack-poisoned]
malfet added a commit that referenced this pull request Dec 30, 2023
@malfet
[MPS] Speedup addmm
9af7b6f 
- Do not copy bias to output
- Skip respective multiplication op if either alpha or beta are equal to 1.0

ghstack-source-id: 836df55
Pull Request resolved: #116548
albanD
albanD approved these changes Jan 1, 2024
View reviewed changes
Copy link
Collaborator

@albanD albanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good!

@malfet
Copy link
Contributor Author

malfet commented Jan 2, 2024

@pytorchbot merge -f "Lint and MPS are green"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@facebook-github-bot facebook-github-bot deleted the gh/malfet/57/head branch January 5, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@albanD albanD albanD approved these changes

@kulinseth kulinseth Awaiting requested review from kulinseth

@Skylion007 Skylion007 Awaiting requested review from Skylion007

Assignees

No one assigned

Labels

ciflow/mps Run MPS tests (subset of trunk) Merged release notes: mps Release notes category topic: improvements topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@malfet @pytorchmergebot @albanD