MLCommons

All

106 repositories

mlperf_automotive
Public
test
Python
•
Apache License 2.0
•546•1•4•1•Updated May 13, 2025May 13, 2025
modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•22•91•321•4•Updated May 13, 2025May 13, 2025
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•546•1.4k•209•28•Updated May 13, 2025May 13, 2025
mlcflow
Public
MLCFlow: Simplifying MLPerf Automations
automation-framework mlperf mlcflow mlperf-automations
Python
•
Apache License 2.0
•10•5•11•0•Updated May 12, 2025May 12, 2025
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•98•403•14•7•Updated May 12, 2025May 12, 2025
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Jupyter Notebook
•
Apache License 2.0
•66•604•149•24•Updated May 12, 2025May 12, 2025
mlperf_inference_test_submissions_v5.0
Public
Mermaid
•5•0•0•0•Updated May 11, 2025May 11, 2025
mlperf-automations
Public
This repository contains automation scripts designed to run MLPerf Inference benchmarks. Originally developed for the Collective Mind (CM) automation framework, these scripts have been adapted to leverage the MLC automation framework, maintained by the MLCommons Benchmark Infrastructure Working Group.
automation mlperf mlperf-inference mlcflow mlperf-automations
Python
•
Apache License 2.0
•12•3•64•1•Updated May 10, 2025May 10, 2025
dynabench
Public
Python
•
MIT License
•17•21•8•4•Updated May 9, 2025May 9, 2025
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•86•176•12•0•Updated May 9, 2025May 9, 2025
mobile_open
Public
MLPerf Mobile benchmarks
Python
•
Apache License 2.0
•2•10•6•1•Updated May 9, 2025May 9, 2025
training_results_v0.5
Public
This repository contains the results and code for the MLPerf™ Training v0.5 benchmark.
Python
•
Apache License 2.0
•52•35•6•6•Updated May 7, 2025May 7, 2025
ck
Public
Collective Knowledge (CK), Collective Mind (CM/CMX) and MLPerf automations: community-driven projects to facilitate collaborative and reproducible research and to learn how to run AI, ML, and other emerging workloads more efficiently and cost-effectively across diverse models, datasets, software, and hardware using MLPerf methodology and benchmarks
metadata education automation optimization best-practices collaboration cm modularity workflows portability
Python
•
Apache License 2.0
•118•627•9•0•Updated May 7, 2025May 7, 2025
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•26•54•31•1•Updated May 7, 2025May 7, 2025
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•35•157•52•22•Updated May 6, 2025May 6, 2025
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•72•382•28•7•Updated May 5, 2025May 5, 2025
mlperf_client
Public
MLPerf Client is a benchmark for Windows and macOS, focusing on client form factors in ML inference scenarios.
benchmarking machine-learning ml
C++
•
Apache License 2.0
•0•13•1•0•Updated Apr 28, 2025Apr 28, 2025
inference_results_v5.0
Public
This repository contains the results and code for the MLPerf™ Inference v5.0 benchmark.
benchmarking benchmark machine-learning
HTML
•4•2•0•1•Updated Apr 22, 2025Apr 22, 2025
training_policies
Public
Issues related to MLPerf™ training policies, including rules and suggested changes
Python
•
Apache License 2.0
•67•95•110•3•Updated Apr 22, 2025Apr 22, 2025
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•48•36•42•1•Updated Apr 18, 2025Apr 18, 2025
ailuminate
Public
The AILuminate v1.1 benchmark suite is an AI risk assessment benchmark developed with broad involvement from leading AI companies, academia, and civil society.
Apache License 2.0
•6•15•20•1•Updated Apr 11, 2025Apr 11, 2025
training
Public
Reference implementations of MLPerf™ training benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•569•1.7k•18•43•Updated Apr 9, 2025Apr 9, 2025
cm4mlperf-results
Public
CM interface and automation recipes to analyze MLPerf Inference, Tiny and Training results. The goal is to make it easier for the community to visualize, compare and reproduce MLPerf results and add derived metrics such as Performance/Watt or Performance/$
visualization training automation inference tiny reproducibility collective-knowledge collective-mind mlperf
Python
•
Apache License 2.0
•3•5•2•0•Updated Apr 9, 2025Apr 9, 2025
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•27•21•10•4•Updated Apr 8, 2025Apr 8, 2025
abtf-ssd-pytorch
Public
Python
•
Apache License 2.0
•1•1•1•2•Updated Apr 8, 2025Apr 8, 2025
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•57•28•35•40•Updated Apr 2, 2025Apr 2, 2025
inference_results_visualization_template
Public template
JavaScript
•
Apache License 2.0
•8•0•1•1•Updated Apr 1, 2025Apr 1, 2025
storage
Public
MLPerf™ Storage Benchmark Suite
performance storage benchmark machine-learning
Shell
•
Apache License 2.0
•41•138•36•4•Updated Mar 29, 2025Mar 29, 2025
submissions_algorithms
Public
Python
•
Apache License 2.0
•3•5•2•1•Updated Mar 25, 2025Mar 25, 2025
mobile_results_v5.0
Public
This repository contains the results and code for the MLPerf™ Mobile Inference v5.0 benchmark.
Apache License 2.0
•0•0•0•0•Updated Mar 21, 2025Mar 21, 2025