Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Synchronize updates; fix AdamW lr_t (keras)
**BUGFIXES**: - Last weight in network would be updated with `t_cur` one update ahead, desynchronizing it from all other weights - `AdamW` in `keras` (optimizers.py, optimizers_225.py) weight updates were _not_ mediated by `eta_t`, so cosine annealing had no effect. Pardon the mishap **FEATURES**: - Added `lr_t` to tf.keras optimizers to track "actual" learning rate externally; use `K.eval(model.optimizer.lr_t)` to get "actual" learning rate for given `t_cur` and `iterations` - Added `lr_t` vs. iterations plot to README, and source code in `example.py` **MISC**: - Added `test_updates` to ensure all weights update synchronously, and that `eta_t` first applies on weights as-is and _then_ updates according to `t_cur` - Fixes #47
- Loading branch information