[dtensor] move pad/unpad_tensor to separate utils #124871
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124871
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit ba282a6 with merge base e592a60 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
bot
added
ci-td-distributed
ciflow/inductor
oncall: distributed
awgu
approved these changes
Apr 24, 2024
|
Looks like need to migrate |
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument cc mrshenli pritamdamania87 zhaojuanmao satgera rohan-varma gqchen aazzolini osalpekar jiayisuse H-Huang kwen2501 awgu penguinwu fegin XilunWu fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k [ghstack-poisoned]
good catch! updated |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 4 jobs have failed, first few of them are: linux-binary-libtorch-cxx11-abi, trunk, linux-binary-manywheel, linux-binary-libtorch-pre-cxx11 Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 4 jobs have failed, first few of them are: linux-binary-libtorch-cxx11-abi, trunk, linux-binary-manywheel, linux-binary-libtorch-pre-cxx11 Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge -i |
|
@pytorchbot revert -m "Broke internal tests, see D56587991 for more details" -c ignoredsignal |
|
@pytorchbot successfully started a revert job. Check the current status here. |
pytorchmergebot
added a commit
that referenced
this pull request
Apr 26, 2024
This reverts commit 0b0eea2. Reverted #124871 on behalf of https://github.com/jeanschmidt due to Broke internal tests, see D56587991 for more details ([comment](#124871 (comment)))
|
@wanchaol your PR has been successfully reverted. |
alat-rights
pushed a commit
to alat-rights/pytorch
that referenced
this pull request
Apr 26, 2024
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument Pull Request resolved: pytorch#124871 Approved by: https://github.com/awgu, https://github.com/wz337
XilunWu
approved these changes
Apr 26, 2024
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument cc mrshenli pritamdamania87 zhaojuanmao satgera rohan-varma gqchen aazzolini osalpekar jiayisuse H-Huang kwen2501 awgu penguinwu fegin XilunWu fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k [ghstack-poisoned]
pytorchmergebot
pushed a commit
that referenced
this pull request
Apr 29, 2024
as titled, we implement a dedicated communication op to allow efficient sharding dimension change using alltoall, to replace our previous allgather + local chunk Pull Request resolved: #124872 Approved by: https://github.com/XilunWu, https://github.com/yifuwang ghstack dependencies: #124871
pytorchmergebot
pushed a commit
that referenced
this pull request
Apr 29, 2024
as titled, as we have a dedicated comm op, this is not needed anymore Pull Request resolved: #124879 Approved by: https://github.com/XilunWu, https://github.com/wz337 ghstack dependencies: #124871, #124872
pytorchmergebot
pushed a commit
that referenced
this pull request
Apr 30, 2024
as titled, we implement a dedicated communication op to allow efficient sharding dimension change using alltoall, to replace our previous allgather + local chunk Pull Request resolved: #124872 Approved by: https://github.com/XilunWu, https://github.com/yifuwang ghstack dependencies: #124871
pytorchmergebot
pushed a commit
that referenced
this pull request
Apr 30, 2024
as titled, as we have a dedicated comm op, this is not needed anymore Pull Request resolved: #124879 Approved by: https://github.com/XilunWu, https://github.com/wz337 ghstack dependencies: #124871, #124872
pytorch-bot bot
pushed a commit
that referenced
this pull request
May 3, 2024
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument Pull Request resolved: #124871 Approved by: https://github.com/awgu, https://github.com/wz337
pytorch-bot bot
pushed a commit
that referenced
this pull request
May 3, 2024
This reverts commit 0b0eea2. Reverted #124871 on behalf of https://github.com/jeanschmidt due to Broke internal tests, see D56587991 for more details ([comment](#124871 (comment)))
petrex
pushed a commit
to petrex/pytorch
that referenced
this pull request
May 3, 2024
as titled, 1. pad/unpad is a general util not specific to the Shard placement, 2. for the propose of the next PR, move these two out of Shard placement itself, and give additional pad_dim argument Pull Request resolved: pytorch#124871 Approved by: https://github.com/awgu, https://github.com/wz337, https://github.com/XilunWu
pytorch-bot bot
pushed a commit
that referenced
this pull request
May 3, 2024
as titled, we implement a dedicated communication op to allow efficient sharding dimension change using alltoall, to replace our previous allgather + local chunk Pull Request resolved: #124872 Approved by: https://github.com/XilunWu, https://github.com/yifuwang ghstack dependencies: #124871
petrex
pushed a commit
to petrex/pytorch
that referenced
this pull request
May 3, 2024
as titled, as we have a dedicated comm op, this is not needed anymore Pull Request resolved: pytorch#124879 Approved by: https://github.com/XilunWu, https://github.com/wz337 ghstack dependencies: pytorch#124871, pytorch#124872
petrex
pushed a commit
to petrex/pytorch
that referenced
this pull request
May 3, 2024
as titled, we implement a dedicated communication op to allow efficient sharding dimension change using alltoall, to replace our previous allgather + local chunk Pull Request resolved: pytorch#124872 Approved by: https://github.com/XilunWu, https://github.com/yifuwang ghstack dependencies: pytorch#124871
pytorch-bot bot
pushed a commit
that referenced
this pull request
May 3, 2024
as titled, as we have a dedicated comm op, this is not needed anymore Pull Request resolved: #124879 Approved by: https://github.com/XilunWu, https://github.com/wz337 ghstack dependencies: #124871, #124872
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
as titled, 1. pad/unpad is a general util not specific to the Shard
placement, 2. for the propose of the next PR, move these two out of Shard
placement itself, and give additional pad_dim argument
cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k