Demo Caffe crowd benchmarking

Grigori Fursin edited this page May 14, 2018 · 5 revisions

[ Home ]

Designing and optimizing computer systems in terms of performance, energy consumption, accuracy, reliability and other metrics has become extremely complex and costly due to an ever growing number of design and optimization choices, constantly changing SW and HW, lack of common performance analysis and optimization methodology, and lack of common ways to reuse and grow optimization knowledge. Failing to optimize properly, however, leads to overly-expensive, under-performing and energy-hungry systems. Very soon there won't be any other way to develop highly efficient systems than by leveraging community effort.

Together with a growing community community across industry and academia, we are developing Collective Knowledge (CK), an open framework and repository for collecting, sharing, reproducing and reusing knowledge about system design and optimization using real-world workloads.

Here we demonstrate how to enable crowd-benchmarking and crowd-tuning of Caffe, a popular Deep Learning framework.

NB: If you already have successfully used Caffe on your system, the following instructions should just work. Otherwise, please install Caffe dependencies first. If you have any issues, please consult the README as well.

First, install CK as briefly described here.

Next, pull the CK-Caffe repository:

 $ ck pull repo --url=

Now you should be able to automatically install Caffe with dependencies such as BLAS libraries and data sets, and start participating in crowd-benchmarking simply by e.g.:

 $ ck crowdbench caffe --user={your email or ID to acknowledge contributions} --env.CK_CAFFE_BATCH_SIZE=2

You can see continuously aggregated results in the public Collective Knowledge repository under the 'crowd-benchmark Caffe library' scenario.

Please note that CK is an on-going, heavily evolving, community-driven long-term project to enable collaborative and systematic benchmarking and tuning of realistic workloads across diverse hardware! Please join the community!

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.