[CUDA][cuBLASLt] addmm -- extend bias fusions to cases with (1 by n) shapes #166307
Conversation
nikitaved
requested review from
IvanYashchuk,
drisspg,
lezcano and
slayton58
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166307
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 2e05c55 with merge base 030de07 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
bot
added
ciflow/b200
ciflow/h100
ciflow/rocm
nikitaved
requested review from
eqy
and removed request for
IvanYashchuk and
lezcano
nikitaved
added
ciflow/trunk
aten/src/ATen/native/cuda/Blas.cpp
Outdated
| && ( | ||
| self.is_contiguous() && | ||
| // NOTE: fine to have 1-len dims to the left from the leading one | ||
| self.dim() <= result.dim() && self.squeeze().dim() == 1 && |
| self._test_addmm_addmv(func, M, m1, m2, activation=activation) | ||
|
|
||
| # vector-shaped bias and beta=1 result in epilogue fusion in CUDA | ||
| # vector-shaped bias (or with 1-len dims on the left from the leading dim) |
There was a problem hiding this comment.
is this an "or" or have we changed the case from vector-shaped bias to 1 x n one?
There was a problem hiding this comment.
It is an or. Motivated by the "expected fusions" tests from Inductor -- and we can fuse these broadcast biases safely.
|
@eqy, thanks for the review! Do you mean in |
|
@eqy, the testing is expanded. Let me know if there is anything else we'd rather do before I merge. |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…shapes (#166307) Pull Request resolved: #166307 Approved by: https://github.com/eqy
Stack from ghstack (oldest at bottom):
cc @ptrblck @msaroufim @eqy @jerryzh168