On the Variance of the Adaptive Learning Rate and Beyond
-
Updated
Jul 31, 2021 - Python
On the Variance of the Adaptive Learning Rate and Beyond
Learning Rate Warmup in PyTorch
ADAM - A Question Answering System. Inspired from IBM Watson
RAdam implemented in Keras & TensorFlow
Pytorch LSTM RNN for reinforcement learning to play Atari games from OpenAI Universe. We also use Google Deep Mind's Asynchronous Advantage Actor-Critic (A3C) Algorithm. This is much superior and efficient than DQN and obsoletes it. Can play on many games
Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch
Easy-to-use AdaHessian optimizer (PyTorch)
Toy implementations of some popular ML optimizers using Python/JAX
Partially Adaptive Momentum Estimation method in the paper "Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks" (accepted by IJCAI 2020)
Quasi Hyperbolic Rectified DEMON Adam/Amsgrad with AdaMod, Gradient Centralization, Lookahead, iterative averaging and decorrelated Weight Decay
[Python] [arXiv/cs] Paper "An Overview of Gradient Descent Optimization Algorithms" by Sebastian Ruder
Adam, NAdam and AAdam optimizers
AdaShift optimizer implementation in PyTorch
implementation of neural network from scratch only using numpy (Conv, Fc, Maxpool, optimizers and activation functions)
This is an implementation of Adam: A Method for Stochastic Optimization.
Tensorflow-Keras callback implementing arXiv 1712.07628
Add a description, image, and links to the adam topic page so that developers can more easily learn about it.
To associate your repository with the adam topic, visit your repo's landing page and select "manage topics."