Portable and customizable Collective Knowledge workflows for CNTK:
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.cm
docker
package
program
soft
.ckr.json
.gitignore
CHANGES
CONTRIBUTIONS
COPYRIGHT
LICENSE
README.md

README.md

Collective Knowledge repository for CNTK

logo logo License

Introduction

This repository provides high-level, portable and customizable workflows for CNTK. It is a part of our long-term community initiative to unify and automate AI, ML and systems R&D using Collective Knowledge Framework (CK), and to collaboratively co-design efficient SW/HW stack for AI/ML during open ACM ReQuEST competitions as described in the ACM ReQuEST report. All benchmarking and optimization results are available in the public CK repository. See CK getting started guide for more details about CK.

Coordination of development

Minimal CK installation

The minimal installation requires:

  • Python 2.7 or 3.3+ (limitation is mainly due to unitests)
  • Git command line client.

Linux/MacOS

You can install CK in your local user space as follows:

$ git clone http://github.com/ctuning/ck
$ export PATH=$PWD/ck/bin:$PATH
$ export PYTHONPATH=$PWD/ck:$PYTHONPATH

You can also install CK via PIP with sudo to avoid setting up environment variables yourself:

$ sudo pip install ck

Windows

First you need to download and install a few dependencies from the following sites:

You can then install CK as follows:

 $ pip install ck

or

 $ git clone https://github.com/ctuning/ck.git ck-master
 $ set PATH={CURRENT PATH}\ck-master\bin;%PATH%
 $ set PYTHONPATH={CURRENT PATH}\ck-master;%PYTHONPATH%

CNTK CK workflow installation (Linux or Windows)

Prerequisites

Linux:

  • If problem with ligpng:
wget -q -O /tmp/libpng12.deb http://mirrors.kernel.org/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1_amd64.deb   && sudo dpkg -i /tmp/libpng12.deb   && rm /tmp/libpng12.deb

CPU

$ ck pull repo:ck-cntk
$ ck install package --tags=lib,cntk,vcpu

GPU including 1bit SGD

$ ck pull repo:ck-cntk
$ ck install package --tags=lib,cntk,vgpu

Checking classification example (and automatically installing available MXNet model(s) via CK)

$ ck run program:cntk
  • Select 'demo-feed-forward-net' command line
  • Observe result

Trying CK CNTK in Docker

Build Docket image:

$ ck build docker:ck-cntk-py35 --sudo

You can now run this Docker image:

$ ck run docker:ck-cntk-py35 --sudo
$ ck run program:cntk

Skip --sudo if you have local Docker installation.

Related Publications with long term CK vision

@inproceedings{ck-date16,
    title = {{Collective Knowledge}: towards {R\&D} sustainability},
    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
    year = {2016},
    month = {March},
    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}

@inproceedings{cm:29db2248aba45e59:0c7348dfbadd5b95,

   author = {{Moreau}, Thierry and {Lokhmotov}, Anton and {Fursin}, Grigori},
    title = "{Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments}",
  journal = {ArXiv e-prints},
archivePrefix = "arXiv",
   eprint = {1801.06378},
 primaryClass = "stat.ML",
 keywords = {Statistics - Machine Learning, Computer Science - Learning, Computer Science - Software Engineering},
     year = 2018,
    month = jan,
   adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180106378M},
  adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

Feedback