Customizable cross-platform package and environment manager with automatic detection, installation and coexistence of multiple versions of software including libraries, compilers, tools, data sets across diverse Linux, Windows, MacOS and Android-based hardware to support collaborative and reproducible CK research workflows:
Switch branches/tags
Nothing to show
Clone or download
Permalink
Failed to load latest commit information.
.cm example of a "static env" entry Jul 24, 2018
cfg added contributors Sep 12, 2018
demo moved platform.init entries from ck-autotuning here; added windows to… Apr 15, 2016
env generate quasi-unique env.variable to detect that the script has alre… Aug 1, 2018
module a new misc.download() function to download a batch of remote entries … Sep 24, 2018
os added support for os:android26* android27* and android28*; improved d… Jun 1, 2018
package fixed clang 7.0 meta Sep 24, 2018
platform.init Update odroid scripts: change to current dir for ondemand and conserv… Sep 6, 2018
script Archive (save) CK repositories with .git info (--all). Sep 20, 2018
soft Automatically detect version for soft:lib.lapack. Sep 21, 2018
.ckr.json updated 'device' module; added more Android OS descriptions Sep 18, 2016
.gitignore Detect OS name on Raspberry Pi Dec 20, 2016
.travis.yml testfix: back to xcode 9.2 since it apparently worked Aug 15, 2018
AUTHORS fixing a small bug Aug 7, 2015
CHANGES added packages for LLVM 6.0.1 and 7.0.0 Sep 24, 2018
CONTRIBUTIONS added contributors Sep 12, 2018
COPYRIGHT.txt added deps_summary to env module to prepare recursive summaries of al… Feb 20, 2018
LICENSE.txt fixing names Sep 8, 2015
README.md added the Appveyor shield (build status image) Aug 15, 2018
appveyor.yml testfix: switched to a different package that compiles under Windows Aug 15, 2018
requirements.txt experimental run of the repository-wide test suite Aug 14, 2018

README.md

Linux & MacOS: Travis Build Status Windows: AppVeyor Build status

Fighting software and hardware chaos in research

logo logo License

This is a Collective Knowledge repository providing functionality for portable, customizable, eproducible and automated experimental workflows. It lets users automatically detect multiple versions of different software (compilers, libraries, tools, models, data sets) using CK software detection plugins or install missing packages in a unified way across diverse hardware with Linux, Windows, MacOS and Android operating systems. It also allows users to collect information about a given platform in a unified way.

Further info:

Contributors

Acknowledgments

logo logo logo logo logo logo logo logo logo logo logo logo logo

License

  • BSD, 3-clause

Minimal CK installation

The minimal installation requires:

  • Python 2.7 or 3.3+ (limitation is mainly due to unitests)
  • Git command line client.

Linux/MacOS

You can install CK in your local user space as follows:

$ git clone http://github.com/ctuning/ck
$ export PATH=$PWD/ck/bin:$PATH
$ export PYTHONPATH=$PWD/ck:$PYTHONPATH

You can also install CK via PIP with sudo to avoid setting up environment variables yourself:

$ sudo pip install ck

Windows

First you need to download and install a few dependencies from the following sites:

You can then install CK as follows:

 $ pip install ck

or

 $ git clone https://github.com/ctuning/ck.git ck-master
 $ set PATH={CURRENT PATH}\ck-master\bin;%PATH%
 $ set PYTHONPATH={CURRENT PATH}\ck-master;%PYTHONPATH%

Installation of a CK workflow for virtual environments and packages

 $ ck pull repo:ck-env

 $ ck list soft
 $ ck list package

Usage

You can easily detect and register in the CK all the instances of GCC and LLVM as follows:

 $ ck detect soft:compiler.gcc
 $ ck detect soft:compiler.llvm

You can now see multiple versions of the detected software registered in the CK as follows:

 $ ck show env

You can then compile and run unified CK benchmarks shared by the community using any of the above compiler instances (GCC, LLVM, ICC ...) and their versions simply as follows:

 $ ck pull repo:ck-autotuning
 $ ck pull repo:ctuning-programs

 $ ck compile program:cbench-automotive-susan --speed
 $ ck run program:cbench-automotive-susan

If you have Android NDK and SDK installed, CK can automatically detect it together with compiler versions (GCC, LLVM), register them and let you compile and run benchmarks on Android simply via:

 $ ck compile program:cbench-automotive-susan --speed --target_os=android21-arm-v7a
 $ ck run program:cbench-automotive-susan --target_os=android21-arm-v7a

You can find further details about our customizable and cross-platform package/environment manager here.

Publications

The concepts have been described in the following publications:

@inproceedings{ck-date16,
    title = {{Collective Knowledge}: towards {R\&D} sustainability},
    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
    year = {2016},
    month = {March},
    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}

All CK-related publications: link

Questions and comments

You are welcome to get in touch with the CK community if you have questions or comments!