Shared repos

Grigori Fursin edited this page Jun 18, 2018 · 36 revisions
Clone this wiki locally

[ Home ]

If you do not know CK, please check CK ReadME and Getting Started Guides

This page has been deprecated. Automatically generated list of all CK repositories and their dependencies is now available at http://cKnowledge.org/shared-repos.html !










Table of Contents

Brief installation notes

Here you can find a manually updated list of shared CK repositories.

You can easily install any Git based repository via

 $ ck pull repo --url=[URL to a GIT-based CK repository]

If CK-powered repository is shared via https://github.com/ctuning, you can easily install it by name:

 $ ck pull repo:[repo name]

For example, you can pull CK repository for universal and multi-objective autotuning via

 $ ck pull repo:ck-autotuning

By default, all pulled repositories are stored in a USER space in the CK directory. If required, this directory can be changed via CK_REPOS environment variable.

Note, that we use https protocol by default. However, if you would like to use git protocol instead, just add --git flag to the above command line.

It is possible to update any shared repository using the same pull command. It is also possible to automatically update all shared Git-based CK repositories via:

 $ ck pull all

You can list all new entries from a pulled repository via:

 $ ck list [repo_name]:: --all

Note, that it is possible to share CK repositories as zip archives (useful to share artifacts along with publications and add them as supplementary material for ACM Digital Library, for example). Such repositories can be installed via

 $ ck add repo:[repo_name] --zip=[zip archive name or full URL if not at github.com/ctuning] --quiet

Finally, you can find all shared modules from CK repositories Shared-modules.

Main CK repositories for universal performance analysis, crowd-tuning and machine learning

  • ck-env (ck pull repo:ck-env) - enable package installation across different platforms (hardware, OS, libs) and simplify co-existence of multiple versions of tools and libraries during experimentation.
  • ck-analytics (ck pull repo:ck-analytics) - CK experimentation, statistical analysis, predictive analytics and graphs.
  • ck-docker (ck pull repo:ck-docker) - automation of various Docker tasks (build, run, pull) via CK with examples.
  • ck-web (ck pull repo:ck-web) - CK web front-end for interactive articles, graphs and tables (see example of our live CK-powered repo here).
  • ck-autotuning (ck pull repo:ck-autotuning) - customizable multi-objective program autotuning/benchmarking (OpenCL/CUDA/MPI/algorithm/compiler flags/etc) and design space exploration for heterogeneous multicore systems (see DATE'16 paper and interactive report).
  • ck-crowdtuning (ck pull repo:ck-crowdtuning) - universal engine to crowdsource experiments across various platforms (cloud, mobile phones, tablets, laptops, data centers, etc) such as multi-objective autotuning (OpenCL, compiler flags, etc).
  • ck-crowdtuning-platforms (ck pull repo:ck-crowdtuning-platforms) - descriptions of platforms participated in experiment crowdsourcing (automatically and manually added features for predictive analytics).
  • ctuning-programs (ck pull repo:ctuning-programs) - CK repository with benchmarks and codelets shared by the community (to move from rigid, outdated and non representative benchmarks to continuously evolving and realistic applications and kernels).
  • ctuning-datasets-min (ck pull repo:ctuning-datasets-min) - CK repository with a minimal set of shared data sets (for above programs).
  • ck-dissemination-modules (ck pull repo:ck-dissemination-modules) - modules for artifact evaluation and knowledge dissemination.

Portable, customizable and reusable CK workflows from the community

AI/ML workflows:

Workflows and artifacts from international projects and ACM/IEEE conferences:

  • reproduce-milepost-project (ck pull repo:reproduce-milepost-project) - artifacts from our past MILEPOST project to extract program semantic features and reproduce machine-learning based compilation.
  • reproduce-carp-project (ck pull repo:reproduce-carp-project) - artifacts (code and data) from the EU FP7 CARP project (polyhedral optimization).
  • reproduce-pamela-project (ck pull repo:reproduce-pamela-project) - artifacts (code and data) and experimental workflows to build and run various SLAM implementations across diverse devices using CK framework!
  • reproduce-ck-paper (ck pull repo:reproduce-ck-paper) - misc artifacts from our CPC'15 paper on performance- and cost-aware software engineering (not yet completed).

KDataSets from our PLDI'10 paper to support computer systems research

Grigori Fursin shared all data sets from the PLDI'10 paper "Evaluating iterative optimization across 1000 datasets" in the Collective Knowledge Format via Google Drive:

List of repositories shared via BitTorrent

We suggest to share large CK repositories in zip archive via BitTorrent to optimize sharing (upload and download) of such repositories across multiple users. We use the following file name convention for such repositories: ckr-<repo></repo>-YYYYMMDD.zip.

  • ckr-32332f6412da1be1-20150803.zip (~287Mb) - Data sets (MiDatasets and cDatasets) from our past R&D to crowdsource autotuning and benchmarking (crowd-tuning and crowd-benchmarking). Get torrent here.