[Gradient Compression] Directly let world_size = group_to_use.size()

[wayi1]

Stack from ghstack:

• [Gradient Compression] Directly let world_size = group_to_use.size() #49715 [Gradient Compression] Directly let world_size = group_to_use.size()
• [Gradient Compression] Explicitly restrict the scope of torch.cuda.synchronize to the current device #49711 [Gradient Compression] Explicitly restrict the scope of torch.cuda.synchronize to the current device
• [Gradient Compression] Change wait() to value() in some callbacks of PowerSGD communication hook #49709 [Gradient Compression] Change wait() to value() in some callbacks of PowerSGD communication hook
• [Gradient Compression] Warm-start of PowerSGD #49451 [Gradient Compression] Warm-start of PowerSGD

Address the comment on #49417 (comment)

Differential Revision: D25673997

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 62e1ee7 (more details on the Dr. CI page):

---

• 3/4 failures possibly* introduced in this PR
  • 1/3 non-CircleCI failure(s)
• 1/4 broken upstream at merge base b288aee on Dec 21 from 3:35pm to 9:32pm

---

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_6_gcc5_4_build (1/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git reset --hard 62e1ee76dc8b95350ecdc5018b203efff2d78aa5
HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git merge --allow-unrelated-histories --no-edit --no-ff 7b4a7661d6de659c8423015a2f3e93308eb83850
Auto-merging torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
CONFLICT (content): Merge conflict in torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (2/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git reset --hard 62e1ee76dc8b95350ecdc5018b203efff2d78aa5
HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git merge --allow-unrelated-histories --no-edit --no-ff 7b4a7661d6de659c8423015a2f3e93308eb83850
Auto-merging torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
CONFLICT (content): Merge conflict in torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

---

1 job timed out:

• pytorch_linux_bionic_py3_8_gcc9_coverage_test1

---

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

• pytorch_linux_bionic_py3_8_gcc9_coverage_test1 on Dec 21 from 3:35pm to 9:32pm (5b163e2 - 7e1356d)
  • 🔁 rerun

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 8 times.

[rohan-varma]

LGTM, although might want to rebase since some CI looks like it failed with merge conflicts.

[facebook-github-bot]

This pull request has been merged in 55b431b.