[Resubmission][Gradient Compression] Refactor default_hooks.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future

[wayi1]

Stack from ghstack:

• [Gradient Compression] Check start_PowerSGD_iter > 1 #51427 [Gradient Compression] Check start_PowerSGD_iter > 1
• [Gradient Compression] Allow BatchedPowerSGD to run vanilla allreduce for the first K iterations #51270 [Gradient Compression] Allow BatchedPowerSGD to run vanilla allreduce for the first K iterations
• [Resubmission][Gradient Compression] Refactor default_hooks.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future #51400 [Resubmission][Gradient Compression] Refactor default_hooks.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: D26162333

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 95a8a14 (more details on the Dr. CI page):

---

• 5/5 failures possibly* introduced in this PR
  • 3/5 non-CircleCI failure(s)

---

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_python_doc_build (1/1)

Step: "Doc Build and Push" (full log | diagnosis details | 🔁 rerun)

Jan 31 06:50:39 Makefile:38: recipe for target 'html' failed

Jan 31 06:50:38 
Jan 31 06:50:38 copying static files... ... done
Jan 31 06:50:38 copying extra files... done
Jan 31 06:50:38 dumping search index in English (code: en)... done
Jan 31 06:50:38 dumping object inventory... done
Jan 31 06:50:38 build finished with problems, 1 warning.
Jan 31 06:50:38 /var/lib/jenkins/workspace/docs/src/pytorch-sphinx-theme/pytorch_sphinx_theme/search.html:21: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
Jan 31 06:50:38   <p class="last">
Jan 31 06:50:38 /var/lib/jenkins/workspace/docs/src/pytorch-sphinx-theme/pytorch_sphinx_theme/search.html:24: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
Jan 31 06:50:38   </p>
Jan 31 06:50:39 Makefile:38: recipe for target 'html' failed
Jan 31 06:50:39 make: *** [html] Error 1
Jan 31 06:50:39 ++ code=2
Jan 31 06:50:39 ++ '[' 2 -ne 0 ']'
Jan 31 06:50:39 ++ set +x
Jan 31 06:50:39 =========================
Jan 31 06:50:39 /var/lib/jenkins/workspace/docs/source/notes/broadcasting.rst:6: WARNING: 'any' reference target not found: numpy.doc.broadcasting
Jan 31 06:50:39 =========================
Jan 31 06:50:39 Docs build failed. If the failure is not clear, scan back in the log
Jan 31 06:50:39 for any WARNINGS or for the line build finished with problems
Jan 31 06:50:39 (tried to echo the WARNINGS above the ==== line)

---

1 failure not recognized by patterns:

Job	Step	Action
pytorch_windows_vs2019_py36_cuda10.1_test1	Unknown	🔁 rerun

---

Extra GitHub checks: 1 failed

• Failed: GitHub Actions - clang-tidy

---

ci.pytorch.org: 1 failed

• Failed: pr/pytorch-linux-bionic-rocm3.10-py3.6

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[codecov]

Codecov Report

Merging #51400 (f08cd46) into gh/SciPioneer/51/base (dbfaf96) will decrease coverage by 0.33%.
The diff coverage is 20.00%.

@@                    Coverage Diff                    @@
##           gh/SciPioneer/51/base   #51400      +/-   ##
=========================================================
- Coverage                  80.86%   80.52%   -0.34%     
=========================================================
  Files                       1938     1938              
  Lines                     211184   211182       -2     
=========================================================
- Hits                      170767   170061     -706     
- Misses                     40417    41121     +704     

[facebook-github-bot]

This pull request has been merged in 0831984.