On the Variance of the Adaptive Learning Rate and Beyond
-
Updated
Jul 31, 2021 - Python
#
adam-optimizer
Here are 86 public repositories matching this topic...
Educational deep learning library in plain Numpy.
-
Updated
Jun 21, 2022 - Python
Reproducing the paper "PADAM: Closing The Generalization Gap of Adaptive Gradient Methods In Training Deep Neural Networks" for the ICLR 2019 Reproducibility Challenge
tensorflow
optimization
keras
wide-residual-networks
adam-optimizer
tensorflow-eager
amsgrad
sgd-momentum
padam
-
Updated
Apr 13, 2019 - Python
Toy implementations of some popular ML optimizers using Python/JAX
machine-learning
momentum
gradient-descent
optimization-algorithms
adam
adam-optimizer
jax
optimizers
-
Updated
Jun 20, 2021 - Python
A collection of various gradient descent algorithms implemented in Python from scratch
machine-learning
ai
deep-learning
machine-learning-algorithms
ml
deep-learning-algorithms
gradient-descent
adagrad
rmsprop
gradient-descent-algorithm
adam-optimizer
nesterov-accelerated-sgd
nesterov-momentum
nesterov
momentum-optimization-algorithm
-
Updated
Feb 28, 2023 - Python
A compressed adaptive optimizer for training large-scale deep learning models using PyTorch
hashing
deep-learning
neural-network
pytorch
transformer
imagenet
count-min-sketch
language-model
adagrad
adam-optimizer
sgd-optimizer
count-sketch
sgd-momentum
-
Updated
Nov 26, 2019 - Python
The project aimed to implement Deep NN / RNN based solution in order to develop flexible methods that are able to adaptively fillin, backfill, and predict time-series using a large number of heterogeneous training datasets.
deep-neural-networks
deep-learning
time-series
recurrent-neural-networks
pca-analysis
rnn
adagrad
rmsprop
leaky-relu
adam-optimizer
random-forest-regressor
adamax
-
Updated
Aug 16, 2016 - Python
Lookahead optimizer ("Lookahead Optimizer: k steps forward, 1 step back") for tensorflow
-
Updated
Sep 3, 2019 - Python
[Python] [arXiv/cs] Paper "An Overview of Gradient Descent Optimization Algorithms" by Sebastian Ruder
adadelta
momentum
gradient-descent
optimization-methods
optimization-algorithms
adam
adagrad
rmsprop
gradient-descent-algorithm
stochastic-optimizers
stochastic-gradient-descent
gradient-boosting
adam-optimizer
adamax
stochastic-optimization
batch-gradient-descent
nesterov-accelerated-sgd
amsgrad
nesterov-momentum
nadam
-
Updated
Mar 23, 2019 - Python
A Deep Learning framework for CNNs and LSTMs from scratch, using NumPy.
python
deep-neural-networks
deep-learning
numpy
cnn
recurrent-neural-networks
lstm
dropout
batch-normalization
rnn
feedforward-neural-network
regularization
deeplearning
convolutional-neural-networks
university-course
maxpooling
l2-regularization
adam-optimizer
pooling
l1-regularization
-
Updated
Jun 12, 2023 - Python
Short description for quick search
tensorflow
dropout
batch-normalization
momentum
softmax-regression
l2-regularization
rmsprop
adam-optimizer
mini-batch-gradient-descent
bias-correction
initialization
gradient-check
exponentially-weighted-averages
learning-rate-decay
-
Updated
Jan 31, 2019 - Python
Implemenation of DDPG with numpy only (without Tensorflow)
neural-network
policy-gradient
reinforcement-learning-algorithms
stochastic-gradient-descent
adam-optimizer
deep-deterministic-policy-gradient
actor-critic-algorithm
replay-buffer
target-network
-
Updated
Mar 4, 2018 - Python
A project I made to practice my newfound Neural Network knowledge - I used Python and Numpy to train a network to recognize MNIST images. Adam and mini-batch gradient descent implemented
-
Updated
Mar 12, 2022 - Python
Nadir is a library of bleeding-edge optimisers built for speed and functionality in PyTorch for researchers 💙
machine-learning
optimization
pytorch
lion
adam-optimizer
adamax
sgd-optimizer
amsgrad
adamw
radam
adamp
adabelief
-
Updated
May 1, 2024 - Python
An Educational Framework Based on PyTorch for Deep Learning Education and Exploration
python
deep-neural-networks
deep-learning
neural-network
numpy
genetic-algorithm
optimizer
pytorch
autograd
artificial-intelligence
deep-learning-library
feedforward-neural-network
convolutional-neural-networks
gradient-descent
backpropagation
cupy
adam-optimizer
perceptrons
deep-learning-framework
training-algorithms
-
Updated
Dec 24, 2023 - Python
Generate novel artistic images using neural style transfer algorithm
-
Updated
Nov 29, 2017 - Python
Here in this system it discloses a log analysis method based on deep learning for an intrusion detection system, which includes the following steps: preprocess the acquired logs of different types in the target system; perform log analysis on the preprocessed logs using a clustering-based method; then, encode the parsed log events into digital f…
-
Updated
Apr 23, 2022 - Python
ND-Adam is a tailored version of Adam for training DNNs.
-
Updated
Sep 19, 2018 - Python
implementation of neural network from scratch only using numpy (Conv, Fc, Maxpool, optimizers and activation functions)
deep-learning
neural-network
numpy
cnn
sgd
convolution
gradient-descent
ann
maxpooling
adam
fully-connected-network
adam-optimizer
implementation-from-scratch
sigmoid-activation
relu-activation
-
Updated
Oct 9, 2022 - Python
Improve this page
Add a description, image, and links to the adam-optimizer topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the adam-optimizer topic, visit your repo's landing page and select "manage topics."