THE Deep Learning Benchmarks
Lua Shell
Latest commit 3b53618 Nov 2, 2016 @soumith soumith committed on GitHub Merge pull request #18 from ngimel/fix_multiGPU
fix multi-GPU benchmark
Permalink
Failed to load latest commit information.
torch fix multi-GPU benchmark Nov 2, 2016
LICENSE Initial commit Apr 14, 2016
README.md adding cntk, as requested by MSFT Jul 27, 2016

README.md

deepmark

THE Deep Learning Benchmarks

See: https://github.com/soumith/convnet-benchmarks/issues/101

Come back here on June 15th, 2016.
A bit delayed due to y'know -- a lot of co-ordination among groups.

Networks

Images

Video

Audio

Text

Platform

  • Initially multi-GPU with (1 to 4 titan-X cards)
  • However, multi-machine, custom hardware, other GPU cards such as AMD, CPUs etc. can and should be accommodated, we will work this out after the initial push.

Metrics

  • Round-trip time for 1 epoch of training (will define an epoch size separately for each network)
  • Maximum batch-size that fits (to show and focus on the extra memory consumption that the framework uses)

Frameworks

Everyone who wants to join-in, but I thought an initial set that is important to cover would be:

  • Caffe
  • Chainer
  • CNTK
  • MXNet
  • Neon
  • Theano
  • TensorFlow
  • Torch

Scripts format

Guarantees

  • I will personally to the best of my abilities make sure that the benchmarking is fair and unbiased. The hope is that the community at large will watch these and point-out / fix mistakes.

Governance

  • The benchmarks will be placed at https://github.com/DeepMark/deepmark and other key community members / organizations who want ownership will be welcome to join in proposing new benchmarks that get relevant as the field progresses.