A Deep Learning Meta-Framework and HPC Benchmarking Library
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
deep500
samples TensorFlow executor for native graphs and a sample Jan 5, 2019
tutorials
.gitignore
LICENSE
README.md Update README.md Jan 25, 2019
setup.py bump down version and add PyPI support Jan 2, 2019

README.md

Deep500: A Deep Learning Meta-Framework and HPC Benchmarking Library


(or: 500 ways to train deep neural networks)

Deep500 is a library that can be used to customize and measure anything with deep neural networks, using a clean, high-performant, and simple interface. Deep500 includes four levels of abstraction: (L0) Operators (layers); (L1) Network Evaluation; (L2) Training; and (L3) Distributed Training.

Using Deep500, you automatically gain:

  • Operator validation, including gradient checking for backpropagation
  • Statistically-accurate performance benchmarks and plots
  • High-performance integration with popular deep learning frameworks (see Supported Frameworks below)
  • Running your operator/framework/optimizer/communicator/... with real workloads, alongside existing environments
  • and much more...

Installation

Using pip: pip install deep500

Usage

See the tutorials.

Requirements

  • Python 3.5 or later
  • Protobuf (sudo apt-get install protobuf-compiler libprotoc-dev)
  • For plotted metrics: matplotlib
  • For distributed optimization:
    • Any MPI implementation (OpenMPI, MPICH, MVAPICH etc.)
    • mpi4py Python package

Supported Frameworks

  • Tensorflow
  • Pytorch
  • Caffe2

Reference

If you use this meta-framework please cite it as:

@inproceedings{deep500,
  author={T. Ben-Nun and M. Besta and S. Huber and A. N. Ziogas and D. Peter and T. Hoefler},
  title={{A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning}},
  year={2019},
  month={May},
  publisher={IEEE},
  note={The 33rd IEEE International Parallel \& Distributed Processing Symposium (IPDPS'19)},
}

Contributing

Deep500 is an open-source, community driven project. We are happy to accept Pull Requests with your contributions!

License

Deep500 is published under the New BSD license, see LICENSE.