A Deep Learning Meta-Framework and HPC Benchmarking Library
Branch: master
Clone or download
Latest commit 024bf50 Jan 25, 2019


Deep500: A Deep Learning Meta-Framework and HPC Benchmarking Library

(or: 500 ways to train deep neural networks)

Deep500 is a library that can be used to customize and measure anything with deep neural networks, using a clean, high-performant, and simple interface. Deep500 includes four levels of abstraction: (L0) Operators (layers); (L1) Network Evaluation; (L2) Training; and (L3) Distributed Training.

Using Deep500, you automatically gain:

  • Operator validation, including gradient checking for backpropagation
  • Statistically-accurate performance benchmarks and plots
  • High-performance integration with popular deep learning frameworks (see Supported Frameworks below)
  • Running your operator/framework/optimizer/communicator/... with real workloads, alongside existing environments
  • and much more...


Using pip: pip install deep500


See the tutorials.


  • Python 3.5 or later
  • Protobuf (sudo apt-get install protobuf-compiler libprotoc-dev)
  • For plotted metrics: matplotlib
  • For distributed optimization:
    • Any MPI implementation (OpenMPI, MPICH, MVAPICH etc.)
    • mpi4py Python package

Supported Frameworks

  • Tensorflow
  • Pytorch
  • Caffe2


If you use this meta-framework please cite it as:

  author={T. Ben-Nun and M. Besta and S. Huber and A. N. Ziogas and D. Peter and T. Hoefler},
  title={{A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning}},
  note={The 33rd IEEE International Parallel \& Distributed Processing Symposium (IPDPS'19)},


Deep500 is an open-source, community driven project. We are happy to accept Pull Requests with your contributions!


Deep500 is published under the New BSD license, see LICENSE.