[pt][group_fusion] fix shape guarding in fusion candidate search #111174
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/111174
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 228cbdf with merge base 898482f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This pull request was exported from Phabricator. Differential Revision: D48696572 |
chaekit
force-pushed
the
export-D48696572
branch
from
2b011e5 to
f34fd52
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48696572 |
…1174) Summary: without the `all` in the fix ``` node.kwargs.get("beta", 1.0) == 1.0 node.kwargs.get("alpha", 1.0) == 1.0 and len(input_shape) == 2 and len(weight_shape) == 2 and all(x % 2 == 0 for x in input_shape + weight_shape) and shape <= MAX_FUSE_TENSOR_SIZE_GROUP_LINEAR # <----- HERE for shape in input_shape + weight_shape ``` this statement defaults to a generator object which means it will always be true. One of the issues is that the shapes could be an odd number which forces gmm to load element-by-element rather than vectorized load. In VDDv3 torchbench example(posted in test plan), you can see there is a 37ms GMM call which swamps any gain from fusion. Overall this change makes the GMM fusion 24% faster Reviewed By: davidberard98 Differential Revision: D48696572
chaekit
force-pushed
the
export-D48696572
branch
from
f34fd52 to
228cbdf
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48696572 |
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
4 participants
Summary:
without the
allin the fixthis statement defaults to a generator object which means it will always be true. One of the issues is that the shapes could be an odd number which forces gmm to load element-by-element rather than vectorized load. In VDDv3 torchbench example(posted in test plan), you can see there is a 37ms GMM call which swamps any gain from fusion. Overall this change makes the GMM fusion 24% faster
Differential Revision: D48696572
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler