Highlighting in the documentation that square root comes before addin…

…g epsilon. #23796
pytorch · Sep 24, 2019 · 12dcaf1 · 12dcaf1
1 parent 04dd9b7
commit 12dcaf1
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/torch/optim/rmsprop.py b/torch/optim/rmsprop.py
@@ -11,6 +11,9 @@ class RMSprop(Optimizer):
     The centered version first appears in `Generating Sequences
     With Recurrent Neural Networks <https://arxiv.org/pdf/1308.0850v5.pdf>`_.
 
+    Note that the implementation here takes the square root of the gradient average before
+    adding epsilon.
+
     Arguments:
         params (iterable): iterable of parameters to optimize or dicts defining
             parameter groups