Farms of CK machines
Clone this wiki locally
[ Home ]
Table of Contents
Researchers often want to validate their ideas and experiments (benchmarking, autotuning, new predictive modeling algorithm, etc) across different machines with different environment (OS, libraries, compilers, etc). However, it is often a very non-trivial task that can easily eat all available research time.
To solve this problem, we provided a module machine in the CK env repository to add, describe and control the status of any available machine as a target for experiments.
It can be any local Linux, MacOS or Windows host, Linux/MacOS machines accessed via SSH, an Android device connected via ADB or ARM's Workload Automation framework, and a Windows based device connected via CK crowdnode.
First, you need to make sure that you have ck-env repository installed:
$ ck pull repo:ck-env
You may also want to update all other repositories (we suggest to do it regularly):
$ ck pull all --kernel
Make sure that your target machine is connected or online (unless it's your host machine).
Just for example, let's consider that you want to use an Android device connected to your host as a target for your experiments.
Now you can describe this new machine as following (let's name it my-android-machine):
$ ck add machine:my-android-machine
You will be asked various questions about the type of this machine. In our case, you need to select 1) CK: Android machine accessed via adb. CK will attempt to automatically detect various properties of your connected device including Android version, device ID, ABI, required SDK, etc, and will eventually add a CK entry machine:my-android-machine.
Note, if you have another Android device, you can add it using the same procedure (just name your target differently).
You can now check the status of all target machines registered in the CK from the command line simply via
$ ck show machine
You can also view their status in your browser:
$ ck browse machine
You can check the status of a give machine via
$ ck check machine:my-android-machine
or in JSON (to use it during automation of experiments):
$ ck check machine:my-android-machine --out=json $ ck check machine --help
It is now possible to use CK widgets platform* to detect various parameters of machines in a farm by specifying --target= flag:
$ ck detect platform.os --target=my-android-machine $ ck detect platform.cpu --target=my-android-machine $ ck detect platform.gpu --target=my-android-machine $ ck detect platform --target=my-android-machine
If you want to use these widgets in your own modules, you can obtain their JSON API using --help flag:
$ ck detect platform --help
You can now easily compile and run any workload shared in CK format (module program) using any target machine from the farm.
Let's consider application for image corner detection (often used in automotive industry):
$ ck list program $ ck list program:*susan
If you want to compile it for Android device, do not forget to download and install Android NDK and Android SDK (CK will then automatically detect it on your Linux/MacOS or Windows machine):
Now you can compile this benchmark simply as following:
$ ck compile program:cbench-automotive-susan --speed --target=my-android-machine
In case of successful compilation, you can now run it on your device as following
$ ck run program:cbench-automotive-susan --target=my-android-machine
Note, that you can compile and run program in one go via CK universal program pipeline (the one which is used in autotuning, crowd-tuning and crowd-benchmarking):
$ ck pipeline program:cbench-automotive-susan --speed --target=my-android-machine
Above organization allows us to reuse CK-based universal, multi-dimensional and multi-objective autotuning plugins (1) for any machine in your farm as following:
$ ck autotune program:cbench-automotive-susan --target=my-android-machine
$ ck crowdtune program:cbench-automotive-susan --target=my-android-machine
If you have questions or feedback, do not hesitate to get in touch with the community via CK mailing list.