Unifying AI API

Grigori Fursin edited this page Aug 30, 2018 · 10 revisions

[ Home ]

Table of Contents

Introduction

Numerous open-source DNN frameworks are nowadays available for the community: TensorFlow, MXNet, Caffe, Caffe2, Torch, Theano, CNTK.

Most of them are rapidly evolving, have different APIs and are incompatible with each other, implemented in different languages, and dramatically differ in performance and usability.

It is often a very difficult choice for end-users to select an appropriate DNN engine which can run on a required hardware (from supercomputers to highly constrained mobile and IoT devices with limited memory, compute power, and energy budget) and with required speed and accuracy.

Unifying AI API using Collective Knowledge Framework

Having a background in physics, machine learning and computer engineering (see CGO test of time award for our interdisiplinary R&D) we decided to solve these problems by providing a simple high-level JSON API for multiple DNN engines, and to collaboratively optimize shared workloads across diverse hardware, models and inputs using open-source Collective Knowledge workflow framework or Android app.

We also let the community continuously share unexpected behavior as well as wrong predictions to build collaborative and realistic training sets rather than using the same and quite outdated ones!

Collective Knowledge supports Python, C, C++, Java and Fortran interfaces and can be used completely locally (all information is kept on a user computer or in a workgroup) or via public repository of knowledge.

The community can extend CK APIs and add other AI scenarios using CK modules.

You can see a public demo of such CK-powered AI services here or by using our Android app.

You can also check it locally as follows:

 $ ck pull repo:ck-analytics
 $ ck browse advice

Examples of CK AI API

You can find different demos of CK AI API in the CK entry "demo:ask-advice-via-ck-ai":

 $ ck find demo:ask-advice-via-ck-ai

DNN-based Image classification

We provided a simple interface to our CK-powered DNN engines and models to a) hide all the underneath complexity, b) protect users from continuous API changes, c) continuously optimize available DNN engines and models across diverse platforms d) create realistic training sets (see cKnowledge.org/ai for more details.

As a proof-of-concept, we let users classify images either using our CK AI DNN engines in a cloud, or using local service from Python as follows:

import ck.kernel as ck

image='use_dnn_to_classify_image.jpg'

r=ck.access({'action':'ask',
             'module_uoa':'advice',
             'to':'classify_image',
             'image':image})
if r['return']>0: ck.err(r)

We continue working with the community to a) improve API and provide more use cases b) optimize various models and Caffe/TensorFlow/Caffe2 engines in terms of accuracy, execution time, energy usage, memory utilization and so on across diverse platforms from IoT to supercomputers. If you are interested, join this collaborative effort.

Machine learning based compiler optimizations

Some years ago we developed MILEPOST GCC which used machine learning to predict compiler optimizations (1, 2, 3).

We recently converted the whole infrastructure to the [Collective] and connected it with collaborative GCC/LLVM optimization (see paper, GCC stats, LLVM stats).

You can now predict optimizations based on MILEPOST features from Python simply as follows:

 import ck.kernel as ck

 scenario='experiment.tune.compiler.flags.gcc.e'
 compiler='GCC 7.1.0'
 # GCC autotuning stats for RPi3
 cpu_name='BCM2709'

 features= ["9.0", "4.0", "2.0", "0.0", "5.0", "2.0", "0.0", "4.0", "0.0", "0.0", "2.0", "0.0", "7.0", "0.0", "0.0", 
            "10.0", "0.0", "0.0", "1.0", "2.0", "10.0", "4.0", "1.0", "14.0", "2.0", "0.714286", 
            "1.8", "3.0", "0.0", "4.0", "0.0", "3.0", "0.0", "3.0", "0.0", "0.0", "0.0", "0.0", "0.0", "0.0", "2.0", 
            "0.0", "1.0", "0.0", "1.0", "5.0", "10.0", "2.0", "0.0", "32.0", "0.0", "10.0", "0.0", "0.0",
            "0.0", "0.0", "3.0", "33.0", "12.0", "32.0", "93.0", "14.0", "19.25", "591.912", "11394.3"]

 r=ck.access({'action':'ask',
              'module_uoa':'advice',
              'to':'predict_compiler_flags',
              'scenario':scenario,
              'compiler':compiler,
              'cpu_name':cpu_name,
              'features':features})
 if r['return']>0: ck.err(r)

 ck.out('Predicted optimization: '+r['predicted_opt'])

CK API in other languages

Besides Python, you can access CK AI API from C, C++, Fortran, Java and PHP using OpenME interface.

Questions and comments

You are welcome to get in touch with the CK community if you have questions or comments!

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.