Collective Knowledge is a cross-platform customizable Python framework to share artifacts as reusable components with JSON API; quickly prototype experimental workflows (such as multi-objective DNN optimization); automate software installation; crowdsource and reproduce experiments; unify predictive analytics; enable interactive articles. Demo:
Python PHP Other
Latest commit f2c5753 Feb 18, 2017 @gfursin gfursin fixing news

README.md

License Linux/MacOS: Build Status Windows: Windows Build status Coverage: Coverage Status

NEWS

Introduction

Collective Knowledge is our "swiss knife" for open, collaborative and reproducible experimentation. CK is a small, portable and customizable research platform to

  • share artifacts as reusable and indexable Python components with unified JSON API and meta information (programs, benchmarks, data sets, tools, predictive models, etc);
  • quickly prototype experimental workflows from shared components (such as customizable and multi-objective autotuning for DSL, OpenCL, CUDA, MPI, OpenMP and compiler flags) across diverse hardware and software;
  • crowdsource experiments across diverse hardware and workloads provided by volunteers, and report "interesting" or unexpected behavior;
  • unify and abstract access to continuously evolving software across Windows, Linux and Android (tools, programs, libraries);
  • protect experimental setups from continuous software and hardware changes;
  • use the latest environment for experimental workflows (when Docker or VirtualBox are not suitable);
  • automate, reproduce and crowdsource empirical experiments (using CK JSON-based web services);
  • unify access to predictive analytics via unified JSON API and CK web services (scikit-learn, R, DNN, etc);
  • enable reproducible and interactive articles.

Project homepage:

Notable use cases:

CK concepts by Michel Steuwer:

License

  • Permissive 3-clause BSD license. (See LICENSE.txt for more details).

Minimal installation

The minimal installation requires:

  • Python 2.7 or 3.3+ (limitation is mainly due to unitests)
  • Python PIP (if you would like to install CK via PIP)
  • Git command line client.

On Ubuntu, you can install these dependencies via

$ apt-get install -y python python-pip git

On Windows, you can download and install these tools from the following sites:

You can now install stable CK version via PIP simply as following (you may need to prefix it with "sudo" on Linux):

$ pip install ck

Alternatively, you can install development CK version in your local user space via GIT as following:

 $ git clone https://github.com/ctuning/ck.git ck

and then add CK to PATH on Linux as following:

 $ export PATH=$PWD/ck/bin:$PATH

or on Windows as following:

 $ set PATH={CURRENT PATH}\ck\bin;%PATH%

Experiment crowdsourcing example

You can now test CK by pulling and executing one of multiple shared research scenarios.

 $ ck pull repo:ck-crowdtuning
 $ ck crowdsource experiment

For example, you can execute shared workflow for collaborative program optimization with all related artifacts, and start participating in multi-objective crowdtuning simply as following:

 $ ck crowdtune program

You can also crowd-tune GCC on Windows as following:

 $ ck crowdtune program --gcc --target_os=mingw-64

If you have GCC or LLVM compilers installed, you can start continuously crowd-tune their optimization heuristics in a quiet mode (for example overnight) via

$ ck crowdtune program --llvm --quiet
$ ck crowdtune program --gcc --quiet

This experimental workflow will be optimizing different shared workloads for multiple objectives (execution time, code size, energy, compilation time, etc) using all exposed design and optimization knobs, while sending best performing optimizations to the public CK-based server:

CK server will, in turn, perform on-line learning to classify optimization versus workloads which can be useful for compiler/hardware designers and performance engineers (described in more detail in http://arxiv.org/abs/1506.06256 ).

You can even participate in collaborative experiments using your Android mobile phone by installing the following application from the Google Play Store:

If you want to try something more complex, you can check our cross-platform DNN unification, customized installation and benchmarking via CK with JSON API and web services:

You can find already shared artifacts and repositories here:

Please check out CK getting started guides and CK wiki for further details:

You can also reuse and customize artifacts with CK-based experimental workflows shared at the leading computer systems' conferences:

Trying CK using Docker image

If you would like to try CK without installing it, you can run the following Docker image:

 $ docker run -it ctuning/ck

However, the idea of CK is to be able to rebuild user experimental workflows natively to take advantage of the latest software environment.

Also note that we added Docker automation to CK (to help evaluate artifacts at the conferences, share interactive and reproducible articles or crowdsource experiments for example).

Please check 'ck-docker' repository at GitHub:

 $ ck show repo:ck-docker

You can download and view one of our CK-based interactive and reproducible articles as following:

 $ ck pull repo:ck-docker
 $ ck run docker:ck-interactive-article --browser (--sudo)

Testing CK

You can test CK functionality via

 $ ck run test

Our related initiatives

Plans

Motivation

Authors

Questions/comments/discussions?

Publications

The concepts have been described in the following publications:

If you found CK useful, feel free to reference any of the above publications. You can download all above references in one BibTex file here.

Testimonials and awards

Acknowledgments

CK development is coordinated by the cTuning foundation (non-profit research organization). We thank the EU TETRACOM 609491 Coordination Action for initial funding and dividiti for continuing support. We would also like to thank Microsoft Research program for one-year grant to host the CK public repository in the Azure cloud. We are also extremely grateful to all volunteers for their valuable feedback and contributions.

logo