distributed machine learning benchmark - a public benchmark of distributed ML solvers and frameworks
JavaScript Python HTML Shell Makefile Smarty CSS
Clone or download
Ralf Grubenmann
Ralf Grubenmann Somewhat fixes DNS lookup
Latest commit 69bf6cf Jul 19, 2018

README.rst

mlbench: Distributed Machine Learning Benchmark

Documentation Status

A public and reproducible collection of reference implementations and benchmark suite for distributed machine learning systems. Benchmark for large scale solvers, implemented on different software frameworks & systems. This is a work in progress and not usable so far

Features

  • For reproducibility and simplicity, we currently focus on standard supervised ML, namely classification and regression solvers.
  • We provide reference implementations for each algorithm, to make it easy to port to a new framework.
  • Our goal is to benchmark all/most currently relevant distributed execution frameworks. We welcome contributions of new frameworks in the benchmark suite
  • We provide precisely defined tasks and datasets to have a fair and precise comparison of all algorithms and frameworks.
  • Independently of all solver implementations, we provide universal evaluation code allowing to compare the result metrics of different solvers and frameworks.
  • Our benchmark code is easy to run on the public cloud.
  • Here is an older [design doc](https://docs.google.com/document/d/1jM4zXRDezEJmIKwoDOKNlGvuNNJk5_FxcBrn1mfYp0E/edit#) for this project.

TODO

Everything

Community

About us: See Authors

Mailing list: https://groups.google.com/d/forum/mlbench

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.