fix: Fixes inplace bias-correction

Bias-correction was wrongly performed inplace, which accumulated bias-correction onto next steps. Fixes #1
frgfm · Aug 29, 2019 · 928ac50 · 928ac50
1 parent 4f2125c
commit 928ac50
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/holocron/optim/ralars.py b/holocron/optim/ralars.py
@@ -88,8 +88,8 @@ def step(self, closure=None):
                 bias_correction2 = 1 - beta2 ** state['step']
 
                 # Bias-correction of first & second moments
-                exp_avg.div_(bias_correction1)
-                exp_avg_sq.div_(bias_correction2)
+                exp_avg = exp_avg / bias_correction1
+                exp_avg_sq = exp_avg_sq / bias_correction2
 
                 # Compute length of SMA
                 sma_t = sma_inf - 2 * state['step'] * (1 - bias_correction2) / bias_correction2