We deeply believe in the power of open science and open source to solve the world's most challenging problems.
Following our tedious experience reproducing 150 research papers and validating them in the real world, we started developing this open-source Collective Knowledge technology to provide a common interface to access and reuse all shared knowledge (research projects, experiments, AI/ML models, code and data), facilitate reproducible research, and simplify transfer to production across rapidly evolving models, software, hardware and data as described in our ACM REP'23 keynote.
Collective Knowledge project consists of the following sub-projects:
- Non-intrusive and technology-agnostic Collective Mind automation language (CM) to bridge the growing gap between research and production. It is extended by the community via
- Collective Knowledge Platform (CK Playground) to provide a user-friendly GUI to help the community explore, reproduce and reuse the state-of-the-art AI, ML and Systems research.
The first practical use case for CM language and CK platform is to let everyone from an expert to a child participate in collaborative benchmarking, optimization and validation of the state-of-the-art AI/ML applications across rapidly evolving models, data, software and hardware from different vendors - see our reproducibility and optimization challenges, shared benchmarking and optimization results for ML Systems (performance, accuracy, power consumption, costs) and the leaderboard.
Read our documentation to learn about how our open-source technology can help you.
Join our Discord channel to ask questions, provide feedback and participate in collaborative developments.
2021-2023 MLCommons
This project is supported by MLCommons, cKnowledge.org, cTuning.org, and individual contributors. We thank HiPEAC and OctoML for sponsoring initial development.