Skip to content

Latest commit

 

History

History
54 lines (39 loc) · 2.66 KB

README-extra.md

File metadata and controls

54 lines (39 loc) · 2.66 KB
Raw

About

This portable script converts raw results from the MLPerf™ Training benchmark to the MLCommons CM format for the Collective Knowledge Playground.

The goal is to make it easier for the community to analyze MLPerf results, add derived metrics such as performance/Watt and constraints, and link reproducibility reports.

Aggreaged results are available in this MLCommons repository.

You can see these results at MLCommons CK playground.

Usage

We have tested this portable CM script on Ubuntu.

Install MLCommons CM automation language.

Pull the MLCommons CK repository with automation recipes for interoperable MLOps:

cm pull repo mlcommons@ck

Install repositories with raw MLPerf training benchmark results:

cmr "get git repo _repo.https://github.com/mlcommons/training_results_v1.0" --extra_cache_tags=mlperf-training-results,version-1.0 --branch=master --depth=""
cmr "get git repo _repo.https://github.com/mlcommons/training_results_v1.1" --extra_cache_tags=mlperf-training-results,version-1.1 --branch=main --depth=""
cmr "get git repo _repo.https://github.com/mlcommons/training_results_v2.0" --extra_cache_tags=mlperf-training-results,version-2.0 --branch=main --depth=""
cmr "get git repo _repo.https://github.com/mlcommons/training_results_v2.1" --extra_cache_tags=mlperf-training-results,version-2.1 --branch=main
cmr "get git repo _repo.https://github.com/mlcommons/training_results_v3.0" --extra_cache_tags=mlperf-training-results,version-3.0

You can install private submission repository as follows:

cm run script "get git repo _repo.https://github.com/mlcommons/submissions_training_v3.0" --extra_cache_tags=mlperf-training-results,version-3.0-private --branch=main --depth=4

Convert raw MLPerf training results into CM experiment entries:

cmr "import mlperf training to-experiment"

Visualize results on your local machine via CK playground GUI:

cmr "gui _playground"

These results are also available in the public CK playground.

Contact us

This project is maintained by the MLCommons taskforce on automation and reproducibility. Join our Discord server to ask questions, provide your feedback and participate in further developments.