A PyTorch implementation of the paper "Decoupled Parallel Backpropagation with Convergence Guarantee"
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
__pycache__
models
results
README.md
main.py
sharedtensor.py
utils.py

README.md

Introduction

This repository is the PyTorch code for the paper Decoupled Parallel Backpropagation with Convergence Guarantee" by Zhouyuan Huo, Bin Gu, Qian Yang and Huang, Heng, ICML 2018.

For any question or suggestions, feel free to contact zhouyuan.huo@pitt.edu

Dependencies

  • Python 3.6.2 (Anaconda)
  • PyTorch 0.3.1
  • CUDA 8.0

Run

Clone and cd into the repo directory, run:

python main.py ./data/cifar.python --dataset cifar10 --arch resnet_ddg --epochs 300 --learning_rate 0.01 --schedule 150 225 --gammas 0.1 0.1 --batch_size 128 --manualSeed 2 --depth 110 --splits 2

Result

Experimental results for optimizing ResNet110 on CIFAR10 using DDG.

Note that to obtain the speedup in the paper, the code should be run in a machine with 4 GPUs.

Citation

@article{huo2018decoupled,
  title={Decoupled Parallel Backpropagation with Convergence Guarantee},
  author={Huo, Zhouyuan and Gu, Bin and Yang, Qian and Huang, Heng},
  journal={arXiv preprint arXiv:1804.10574},
  year={2018}
}