Layer-Wise-AAAI20

Code repository for AAAI'20 paper: On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning

CIFAR-10 training using Layerwise and Entire-Model Compressed Gradient Communication

Please check CIFAR10 Folder for the implementation and training scripts

ImageNet training using Layerwise and Entire-Model Compressed Gradient Communication

Please check ImageNet Folder for the implementation and training scripts

Citation

@inproceedings{layer-wise,
    author = {Dutta, Aritra and Bergou, El Houcine and Abdelmoniem, Ahmed M. and Ho, Chen-Yu and Sahu, Atal Narayan and Canini, Marco and Kalnis, Panos},
    title = {{On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning}},
    booktitle={Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20)},
year={2020}
}