Skip to content
Partially Adaptive Momentum Estimation
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
models
LICENSE Create LICENSE Nov 14, 2018
Padam.py fixed factor Sep 21, 2018
README.md new version Jan 23, 2019
run_cnn_test_cifar10.py new version Jan 23, 2019
run_cnn_test_cifar100.py new version Jan 23, 2019
utils.py Padam v1 Jun 5, 2018

README.md

Padam

This repository contains our pytorch implementation of Partially Adaptive Momentum Estimation method (Padam) in the paper [Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks].

Prerequisites:

  • Pytorch
  • CUDA

Usage:

Use python to run run_cnn_test_cifar10.py for experiments on Cifar10 and run_cnn_test_cifar100.py for experiments on Cifar100

Command Line Arguments:

  • --lr: (start) learning rate
  • --method: optimization method, e.g., "sgdm", "adam", "amsgrad", "padam"
  • --net: network architecture, e.g. "vggnet", "resnet", "wideresnet"
  • --partial: partially adaptive parameter for Padam method
  • --wd: weight decay
  • --Nepoch: number of training epochs
  • --resume: whether resume from previous training process

Usage Examples:

  • Run experiments on Cifar10:
  -  python run_cnn_test_cifar10.py  --lr 0.1 --method "padam" --net "vggnet"  --partial 0.125 --wd 5e-4
  • Run experiments on Cifar100:
  -  python run_cnn_test_cifar100.py  --lr 0.1 --method "padam" --net "resnet"  --partial 0.125 --wd 5e-4
You can’t perform that action at this time.