ban fusion of large amount of reads #158667
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158667
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 2 PendingAs of commit 34ffcc2 with merge base 70b4a88 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "topic: not user facing" |
|
@pytorchbot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
This is an reland attempt of pytorch#157563, but insteading of introducing the `realize_acc_reads_size_threshold` config and setting to a default value, we set it to `None` for now to unblock an internal use case. Will deep dive into the issue and harden the logic in later PRs. Pull Request resolved: pytorch#158667 Approved by: https://github.com/yf225
| def fusion_accumulate_large_reads( | ||
| self, node1: BaseSchedulerNode, node2: BaseSchedulerNode, threshold: int | ||
| ) -> bool: | ||
| all_reads = (node1.read_writes.reads | node2.read_writes.reads) - ( |
There was a problem hiding this comment.
Curious that besides reads, will writes also be a problem, will the buffer for writes be hold by the kernel? If a large write buffer is generated by the last node of a fusion, with fusion, will the large write buffer be hold from the beginning of the fusion kernel?
This is to revert some of the changes in #158667 In particular, we only disallow fusion of large accumulate read at IR level and not at scheduler level, as users can create their own custom fusion logics for the scheduler level. Pull Request resolved: #161978 Approved by: https://github.com/yf225
This is to revert some of the changes in pytorch#158667 In particular, we only disallow fusion of large accumulate read at IR level and not at scheduler level, as users can create their own custom fusion logics for the scheduler level. Pull Request resolved: pytorch#161978 Approved by: https://github.com/yf225
This is to revert some of the changes in pytorch#158667 In particular, we only disallow fusion of large accumulate read at IR level and not at scheduler level, as users can create their own custom fusion logics for the scheduler level. Pull Request resolved: pytorch#161978 Approved by: https://github.com/yf225
This is to revert some of the changes in pytorch#158667 In particular, we only disallow fusion of large accumulate read at IR level and not at scheduler level, as users can create their own custom fusion logics for the scheduler level. Pull Request resolved: pytorch#161978 Approved by: https://github.com/yf225
This is to revert some of the changes in pytorch#158667 In particular, we only disallow fusion of large accumulate read at IR level and not at scheduler level, as users can create their own custom fusion logics for the scheduler level. Pull Request resolved: pytorch#161978 Approved by: https://github.com/yf225
4 participants
Stack from ghstack (oldest at bottom):
This is an reland attempt of #157563, but insteading of introducing the
realize_acc_reads_size_thresholdconfig and setting to a default value, we set it toNonefor now to unblock an internal use case. Will deep dive into the issue and harden the logic in later PRs.cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben