[C10D] Add __repr__ to P2POp class #126538
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126538
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 3f99204 with merge base failed to retrieve merge base, please contact dev infra: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
bot
added
oncall: distributed
|
Wonder whether folks prefer local ranks or global ranks for src/dsc I went with local since i thought its helpful to print the PG too and if you print the PG and then print global ranks its confusing. Also, the log-prefix for each log line usually shows the global rank, so you get both pieces of info. And local rank is more 'helpful' in some cases (stage 0 is local rank 0, for PP) otoh our APIs are confusing, and they ask you to specify a PG and also specify global ranks, so maybe printing global ranks is most consistent. |
| else: | ||
| return super().__repr__() | ||
|
|
||
| return f"P2POp({op_name} pg={group_name}, s={s}, d={d}, {self.tensor.shape}, {self.tensor.dtype})" |
There was a problem hiding this comment.
Print the tag too if it exists?
Don't know how/when it's used - but might be useful?
tag (int, optional): Tag to match send with recv.
There was a problem hiding this comment.
yea.... actually NCCL doesn't even support tag. So i would print it out of 'completeness' or compatibility with Gloo, but i think nobody is using tag anyway and we should probably deprecate it.
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Pull Request resolved: pytorch#126538 Approved by: https://github.com/Skylion007, https://github.com/kwen2501, https://github.com/c-p-i-o ghstack dependencies: pytorch#126419
Stack from ghstack (oldest at bottom):
cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @yf225 @chauhang @d4l3k