This documentation explains how to compose, run, customize and extend MLPerf benchmarks in a unified way across diverse models, data sets, software and hardware from different vendors using MLCommons Collective Mind automation recipes:
- MLPerf inference benchmark
- MLPerf training benchmark (prototyping phase)
- MLPerf tiny benchmark (prototyping phase)
- MLPerf automotive (prototyping phase)
- MLPerf mobile (preparation phase)
- MLPerf client (preparation phase)
Note that the MLCommons Task Force on Automation and Reproducibility is preparing a GUI to make it easier to run, customize, reproduce and compare MLPerf benchmarks - please stay tuned for more details!
Don't hesitate to get in touch via the public Discord server if you have questions or feedback!