Skip to content

Commit

Permalink
Update base for Update on "[Gradient Compression] Allow BatchedPowerS…
Browse files Browse the repository at this point in the history
…GD to run vanilla allreduce for the first K iterations"

Similar to #50973, allow the batched version to run vanilla allreduce for the first K iterations.

This may be useful if the batched version can be applied to some use cases where the accuracy requirement is not very strict.

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: [D26077709](https://our.internmc.facebook.com/intern/diff/D26077709/)

[ghstack-poisoned]
  • Loading branch information
wayi committed Jan 31, 2021
2 parents 04be512 + 95a8a14 commit a2a5115
Showing 0 changed files with 0 additions and 0 deletions.

0 comments on commit a2a5115

Please sign in to comment.