doc string fixed in torch.distributed.reduce_scatter #84983

ShisuiUzumaki · 2022-09-14T03:55:07Z

Previous torch.distributed.reduce_scatter:

def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False):
    """
    Reduces, then scatters a list of tensors to all processes in a group.

    Args:
        output (Tensor): Output tensor.
        input_list (list[Tensor]): List of tensors to reduce and scatter.
        group (ProcessGroup, optional): The process group to work on. If None,
            the default process group will be used.
        async_op (bool, optional): Whether this op should be an async op.

Fixed:

def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False):
    """
    Reduces, then scatters a list of tensors to all processes in a group.

    Args:
        output (Tensor): Output tensor.
        input_list (list[Tensor]): List of tensors to reduce and scatter.
        op (optional): One of the values from
            ``torch.distributed.ReduceOp``
            enum.  Specifies an operation used for element-wise reductions
        group (ProcessGroup, optional): The process group to work on. If None,
            the default process group will be used.
        async_op (bool, optional): Whether this op should be an async op.

pytorch-bot · 2022-09-14T03:55:09Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84983

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures, 1 Pending

As of commit c898da4:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

H-Huang

Overall LGTM, I think we may have to update the docs to include enum values for torch.distributed.ReduceOp as well. I am starting CI tests and waiting for docs to get built to confirm the changes before I approve.

torch/distributed/distributed_c10d.py

Co-authored-by: Howard Huang <howardhuang96@gmail.com>

ShisuiUzumaki

ok

ShisuiUzumaki · 2022-09-15T04:04:29Z

@H-Huang Can you merge it now?

H-Huang

LGTM, thanks for adding!

H-Huang · 2022-09-15T15:27:55Z

@pytorchbot merge

pytorchmergebot · 2022-09-15T15:30:35Z

@pytorchbot successfully started a merge job. Check the current status here and land check progress here.
The merge job was triggered with the land checks (-l) flag. If you did not specify this flag yourself, you are likely enrolled in the land checks rollout. This means that your change will be merged once all checks on your PR and the land checks have passed (ETA 4 Hours). If you need to coordinate lands between different changes and cannot risk a land race, please add the ciflow/trunk label to your PR and wait for signal to complete, and then land your changes in proper order. Having trunk, pull, and Lint pre-run on a PR will bypass land checks and the ETA should be immediate. If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

Fixes #84865 Previous `torch.distributed.reduce_scatter`: ``` def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False): """ Reduces, then scatters a list of tensors to all processes in a group. Args: output (Tensor): Output tensor. input_list (list[Tensor]): List of tensors to reduce and scatter. group (ProcessGroup, optional): The process group to work on. If None, the default process group will be used. async_op (bool, optional): Whether this op should be an async op. ``` Fixed: ``` def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False): """ Reduces, then scatters a list of tensors to all processes in a group. Args: output (Tensor): Output tensor. input_list (list[Tensor]): List of tensors to reduce and scatter. op (optional): One of the values from ``torch.distributed.ReduceOp`` enum. Specifies an operation used for element-wise reductions group (ProcessGroup, optional): The process group to work on. If None, the default process group will be used. async_op (bool, optional): Whether this op should be an async op. ``` Pull Request resolved: #84983 Approved by: https://github.com/H-Huang

pytorchmergebot · 2022-09-15T17:41:32Z

Merge failed

Reason: Failed to merge; some land checks failed: trunk, trunk / android-emulator-build-test / build-and-test

If you believe this is an error, you can use the old behavior with @pytorchbot merge -g (optionally with the ciflow/trunk to get land checks) or use @pytorchbot merge -f "some reason here". For more information, see the bot wiki.

Please reach out to the PyTorch DevX Team with feedback or questions!

Details for Dev Infra team

Raised by workflow job

H-Huang · 2022-09-15T18:15:37Z

@pytorchbot merge -g

pytorchmergebot · 2022-09-15T18:17:06Z

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

github-actions · 2022-09-15T18:18:02Z

Hey @ShisuiUzumaki.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

Fixes #84865 Previous `torch.distributed.reduce_scatter`: ``` def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False): """ Reduces, then scatters a list of tensors to all processes in a group. Args: output (Tensor): Output tensor. input_list (list[Tensor]): List of tensors to reduce and scatter. group (ProcessGroup, optional): The process group to work on. If None, the default process group will be used. async_op (bool, optional): Whether this op should be an async op. ``` Fixed: ``` def reduce_scatter(output, input_list, op=ReduceOp.SUM, group=None, async_op=False): """ Reduces, then scatters a list of tensors to all processes in a group. Args: output (Tensor): Output tensor. input_list (list[Tensor]): List of tensors to reduce and scatter. op (optional): One of the values from ``torch.distributed.ReduceOp`` enum. Specifies an operation used for element-wise reductions group (ProcessGroup, optional): The process group to work on. If None, the default process group will be used. async_op (bool, optional): Whether this op should be an async op. ``` Pull Request resolved: #84983 Approved by: https://github.com/H-Huang

doc string fixed in torch.distributed.reduce_scatter

ad44f71

ShisuiUzumaki requested review from mrshenli, zhaojuanmao, pritamdamania87, rohan-varma, mingzhe09088, H-Huang and awgu as code owners September 14, 2022 03:55

pytorch-bot bot added the release notes: distributed (c10d) release notes category label Sep 14, 2022

facebook-github-bot added cla signed oncall: distributed Add this issue/PR to distributed oncall triage queue labels Sep 14, 2022

pytorchbot added the open source label Sep 14, 2022

H-Huang reviewed Sep 14, 2022

View reviewed changes

torch/distributed/distributed_c10d.py Outdated Show resolved Hide resolved

Update torch/distributed/distributed_c10d.py

c898da4

Co-authored-by: Howard Huang <howardhuang96@gmail.com>

ShisuiUzumaki commented Sep 15, 2022

View reviewed changes

H-Huang approved these changes Sep 15, 2022

View reviewed changes

pytorchmergebot added the Merged label Sep 15, 2022

pytorchmergebot closed this in 6bd7d0f Sep 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

doc string fixed in torch.distributed.reduce_scatter #84983

doc string fixed in torch.distributed.reduce_scatter #84983

Uh oh!

ShisuiUzumaki commented Sep 14, 2022

Uh oh!

pytorch-bot bot commented Sep 14, 2022 •

edited

Loading

Uh oh!

H-Huang left a comment

Uh oh!

Uh oh!

ShisuiUzumaki left a comment

Uh oh!

ShisuiUzumaki commented Sep 15, 2022

Uh oh!

H-Huang left a comment

Uh oh!

H-Huang commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Uh oh!

H-Huang commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Uh oh!

github-actions bot commented Sep 15, 2022

Uh oh!

Uh oh!

doc string fixed in torch.distributed.reduce_scatter #84983

doc string fixed in torch.distributed.reduce_scatter #84983

Uh oh!

Conversation

ShisuiUzumaki commented Sep 14, 2022

Uh oh!

pytorch-bot bot commented Sep 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84983

✅ No Failures, 1 Pending

Uh oh!

H-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ShisuiUzumaki left a comment

Choose a reason for hiding this comment

Uh oh!

ShisuiUzumaki commented Sep 15, 2022

Uh oh!

H-Huang left a comment

Choose a reason for hiding this comment

Uh oh!

H-Huang commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Merge failed

Uh oh!

H-Huang commented Sep 15, 2022

Uh oh!

pytorchmergebot commented Sep 15, 2022

Uh oh!

github-actions bot commented Sep 15, 2022

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 14, 2022 •

edited

Loading