Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Farms of CK machines
[ Home ]
Table of Contents
Researchers often want to validate their ideas and experiments (benchmarking, autotuning, new predictive modeling algorithm, etc) across different machines with different environment (OS, libraries, compilers, etc). However, it is often a very non-trivial task that can easily eat all available research time.
To solve this problem, we provided a module machine in the CK env repository to add, describe and control the status of any available machine as a target for experiments.
It can be any local Linux, MacOS or Windows host, Linux/MacOS machines accessed via SSH, an Android device connected via ADB or ARM's Workload Automation framework, and a Windows based device connected via CK crowdnode.
First, you need to make sure that you have ck-env repository installed:
$ ck pull repo:ck-env
You may also want to update all other repositories (we suggest to do it regularly):
$ ck pull all --kernel
Make sure that your target machine is connected or online (unless it's your host machine).
Just for example, let's consider that you want to use an Android device connected to your host as a target for your experiments.
Now you can describe this new machine as follows (let's name it my-android-machine):
$ ck add machine:my-android-machine
You will be asked various questions about the type of this machine. In our case, you need to select CK: Android machine accessed via adb. CK will attempt to automatically detect various properties of your connected device including Android version, device ID, ABI, required SDK, etc, and will eventually add a CK entry machine:my-android-machine.
Note, if you have another Android device, you can add it using the same procedure (just name your target differently).
We recently added support to let Windows-based machines participate in CK experiment crowdsourcing. Simply download CK crowd-node, install and run it on your target Windows device. This is a small application which will set up a server on a Windows machine similar to adb but with CK JSON API. When it is executed, it will print host IP, secret key and other information. You just need to copy secret key to a file on your host machine, and then add this machine to CK farm via
$ ck add machine:my-windows-machineYou will then need to select CK: remote Windows machine accessed via CK crowd node and provide all requested information including path to the file with a secret key.
You can now check the status of all target machines registered in the CK from the command line simply via
$ ck show machine
You can also view their status in your browser:
$ ck browse machine
You can check the status of a give machine via
$ ck check machine:my-android-machine
or in JSON (to use it during automation of experiments):
$ ck check machine:my-android-machine --out=json $ ck check machine --help
It is now possible to use CK widgets platform* to detect various parameters of machines in a farm by specifying --target= flag:
$ ck detect platform.os --target=my-android-machine $ ck detect platform.cpu --target=my-android-machine $ ck detect platform.gpu --target=my-android-machine $ ck detect platform --target=my-android-machine
If you want to use these widgets in your own modules, you can obtain their JSON API using --help flag:
$ ck detect platform --help
You can now easily compile and run any workload shared in CK format (module program) using any target machine from the farm.
Let's consider application for image corner detection (often used in automotive industry):
$ ck list program $ ck list program:*susan
If you want to compile it for Android device, do not forget to download and install Android NDK and Android SDK (CK will then automatically detect it on your Linux/MacOS or Windows machine):
Now you can compile this benchmark simply as follows:
$ ck compile program:cbench-automotive-susan --speed --target=my-android-machine
In case of successful compilation, you can now run it on your device as follows
$ ck run program:cbench-automotive-susan --target=my-android-machine
Note, that you can compile and run program in one go via CK universal program pipeline (the one which is used in autotuning, crowd-tuning and crowd-benchmarking):
$ ck pipeline program:cbench-automotive-susan --speed --target=my-android-machine
Above organization allows us to reuse CK-based universal, multi-dimensional and multi-objective autotuning plugins (1) for any machine in your farm as follows:
$ ck autotune program:cbench-automotive-susan --target=my-android-machine
$ ck crowdtune program:cbench-automotive-susan --target=my-android-machine
You are welcome to get in touch with the CK community if you have questions or comments!