Interactive articles

Grigori Fursin edited this page Jul 13, 2018 · 13 revisions

[ Home ]

Interactive reports

We started providing CK components with JSON API to automatically generate different tables, graphs, static papers and even interactive articles connected with reproducible workflows and experiments.

Most of such components are available in the following CK repositories:

 $ ck pull repo:ck-analytics
 $ ck pull repo:ck-dissemination-modules

See the interactive report automatically generated by CK framework for a project sponsored by the Raspberry Pi foundation, cTuning foundation and dividiti to create a "a Collective Knowledge workflow for collaborative research into multi-objective autotuning and machine learning techniques":

We did not yet have time and resources to automate the creation of such interactive reports and graphs but you can already check how it's done and reuse components from the following repositories used to generate above report:

See CK paper entry, CK report entry, CK graphs, CK scripts, and CK slides.

You can also get in touch with the CK community via mailing list or Slack channel - we regularly help our users there.

Interactive graphs

These notes must be updated.

We attempt to unify various plotting mechanisms in the CK to reduce researchers burden. We have many examples of unified 1D, 2D and 3D plotting in the CK repository ck-analytics via JSON API:

 $ ck pull repo:ck-analytics
 $ ck list ck-analytics:demo:graph*

This approach together with distributed Unique IDs for each entry also makes it relatively straightforward to implement interactive graphs, articles and other objects integrated within HTML pages.

At the moment this functionality is working well but on a low SDK level, i.e. creating such content is not yet user-friendly. However, it should not be difficult to provide extra high-level modules to simplify creation of CK-powered widgets. If you are interested to help, please contact the CK community via this mailing list.

Here, we will just demonstrate a few examples of creating such interactive content based on autotuning examples of an image processing benchmark. These live examples can be found on the following pages:

Above examples are available in the ck-autotuning repository which can be obtained via

$ ck pull repo:ck-crowdtuning

It is possible to perform compiler flag autotuning of cbench-automotive-susan benchmark via scripts from demo:cbench-automotive-susan entry:

$ cd `ck find demo:cbench-automotive-susan`
$ ./_clean_program_pipeline.sh
$ ./_setup_program_pipeline.sh
$ ./autotune_program_pipeline_i100.sh

During this autotuning, experimental results are recorded in the experiment:demo-autotune-flags-susan-linux-i100 entry.

It is now possible to produce and record a graph in PNG format (other formats such as jpeg and PDF are also supported) into graph:demo-autotuning-compiler-flags-susan-linux entry using another script:

$ ./plot_as_points_with_pareto_save_to_file.sh

This script is customized via plot_as_points_with_pareto_save_to_file.json:

 ...
  "out_repo_uoa":"ck-autotuning",
  "out_data_uoa":"demo-autotuning-compiler-flags-susan-linux",
  "out_to_file":"2d_points_time_vs_size_with_pareto.png",
  "save_table_to_json_file":"2d_points_time_vs_size_with_pareto.json",
  "save_table_to_csv_file":"2d_points_time_vs_size_with_pareto.csv"
 }

It specifies where to record PNG file (repo and entry UOA) as well as experimental table and CSV file (useful to rebuild graphs interactively) or allowing the community to process table via their own third-party tools such as Microsoft Excel, MATLAB, OpenOffice, etc.

The non-user friendly part is to prepare a meta description of the graph entry graph:demo-autotuning-compiler-flags-susan-linux to tell CK how to produce interactive HTML - we skip this part and let you check it yourself via:

$ ck load graph:demo-autotuning-compiler-flags-susan-linux

You can now start you local CK web server:

$ ck start web

And then visualize this graph via:

$ firefox http://localhost/ck/repo/web.php?wcid=graph:demo-autotuning-compiler-flags-susan-linux

It is also possible to visualize above experiments as table:

$ firefox localhost/ck/repo/web.php?wcid=experiment:demo-autotune-flags-susan-linux-best

Note, that both modules graph and experiment has an action html_viewer which is used to customize such views, otherwise a universal CK entry viewer will be used such as for datasets:

$ firefox http://localhost/ck/repo/web.php?wcid=8a7141c59cd335f5:b2130844c38e4a56
$ firefox http://localhost/ck/repo/web.php?wcid=dataset:image-pgm-0001

In the table view, it is also possible to customize raws using experiment.view module. Entries has a very simple meta-format, describing which flat keys to show and what is their description:

$ ck load experiment.view:autotuning-basic

 {
  "flat_keys": [
    "##choices#host_os", 
    "##choices#target_os", 
    "##features#platform#cpu#name", 
    "##features#platform#cpu#sub_name"
    ...
  ], 
  "flat_keys_desc": [
    {
      "desc": "Host OS", 
      "module_uoa": "0440cb72c2bc5cc6", 
      "type": "uoa"
    }, 
    {
      "desc": "Target OS", 
      "module_uoa": "0440cb72c2bc5cc6", 
      "type": "uoa"
    }, 
    {
      "desc": "CPU name"
    }, 
    {
      "desc": "CPU sub-name"
    }, 
    ...

This shows that you can not only share artifacts and workflows via CK repositories but also interactive graphs and reports making it much more convenient for the community or reviewers to test/validate your approach.

Furthermore, it is possible even to push output of benchmarks (such as slambench) directly to pages thus sharing efficient live demos and live reports as briefly described here:

Questions and comments

You are welcome to get in touch with the CK community if you have questions or comments!

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.