Make object-related collective functions accept a device parameter

[shaoyf42]

Make object-related collective functions accept a device parameter. Then those functions can support custom device to fix #97938
The next step will be to extend the _get_pg_device for supporting the custom device and backend.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/98130

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 83ee045:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[shaoyf42]

@H-Huang could you take a look

[ezyang]

Sure why not

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

The merge job was canceled. If you believe this is a mistake,then you can re trigger it through pytorch-bot.

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: This PR is too stale; the last push date was more than 3 days ago. Please rebase and try again. You can rebase and merge by leaving the following comment on this PR:
@pytorchbot merge -r
Or just rebase by leaving @pytorchbot rebase comment

Details for Dev Infra team

Raised by workflow job

[ezyang]

@pytorchbot merge -r

[pytorchmergebot]

@pytorchbot successfully started a rebase job. Check the current status here

[pytorchmergebot]

Successfully rebased dev_c10d onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout dev_c10d && git pull --rebase)

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

The merge job was canceled. If you believe this is a mistake,then you can re trigger it through pytorch-bot.

[kwen2501]

Putting a temporary hold here as we were discussing in #97938 (comment) that we may want to move all **_object collectives to cpu only. (In that case, custom backend don't need to implement them.)

[kwen2501]

FYI of similar try:
Figure out device to use for object collectives #100238

[shaoyf42]

Closing as the rebased version #100954 has been merged to main.
A thousand thanks to @huihoaan @kwen2501 !