Skip to content
Branch: master
Find file History
sridhargopinath and harsha-simhadri Feature: Automatic quantization using SeeDot (#88)
* Added SeeDot compiler

* Adding antlr jar file

* Added Predictor project

* Tested on cr-binary

* Updating .proj file

* Updating .gitignore

* Adding Streamer project

* Added code to read profile file

* Updated .pyproj file

* Fixed issue in visitBop2

* Fixed HLS codegen

* Refactored Predictor project

* Deleted files

* Few more deletionS

* Updated gitignore

* Added 2 line copyright prefix

* Updated SeeDot.Antlr and SeeDot.AST

* More changes to the structure of the repo

* Small fix

* More changes

* Minor changes'

* Minor

* More changes

* Minor

* Minor

* Major changes in IRBuilder.py

* Few more

* More changes

* More changes

* First iteration done. Tested on previous commit

* Minor

* Minor change in library.h

* Codegen change

* Minor

* Changes to msbuild path

* Supporting libsvm format

* Minor

* Updated the Predictor project to remove dirs

* Removed windows.h in Predictor

* Added Makefile

* Updated scripts to work with Makefile

* Added support to move intermediate files to scratch directory

* Dir restructuring

* Minor

* PEP8 formatting

* Minor

* Minor 2

* Removed Lenet

* Temp removing files

* New naming convention

* Removed irGen

* Tested on bonsai

* Removed Windows-specific files

* Removed HLS and verilog
:

* Removed workers parameter

* More comments

* Comments and cleaning

* Moved SeeDot

* Updated edgeml refs

* removed examples/seedot

* Deleting init.py

* Updated dir structure

* Rename

* Minor

* Removing requirements

* Updated README. Expects class IDs from 0

* Added support for zero index labels

* Updated README

* Minor

* Readme update:

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Minor change for Bonsai

* Update README.md

* Directory re-org

* Adding pre-loaded models

* Update README.md

* Update README.md

* Minor

* Update README.md

* Update README.md

* Create README.md

* Update README.md

* Update README.md

* Updated python package requirements
Latest commit 84bbb46 May 29, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
seedot Feature: Automatic quantization using SeeDot (#88) May 28, 2019
README.md Feature: Automatic quantization using SeeDot (#88) May 28, 2019
SeeDot.py Feature: Automatic quantization using SeeDot (#88) May 28, 2019

README.md

SeeDot

SeeDot is an automatic quantization tool that generates efficient machine learning (ML) inference code for IoT devices.

Overview

ML models are usually expressed in floating-point, and IoT devices typically lack hardware support for floating-point arithmetic. Hence, running such ML models on IoT devices involves simulating floating-point arithmetic in software, which is very inefficient. SeeDot addresses this issue by generating fixed-point code with only integer operations. To enable this, SeeDot takes as input trained floating-point models (like Bonsai or ProtoNN) and generates efficient fixed-point code that can run on microcontrollers. The SeeDot compiler uses novel compilation techniques like automatically inferring certain parameters used in the fixed-point code, optimized exponentiation computation, etc. With these techniques, the generated fixed-point code has comparable classification accuracy and performs significantly faster than the floating-point code.

To know more about SeeDot, please refer to our publication here.

This document describes the tool usage with an example.

Software requirements

  1. Python 3 with following packages:
    • Antrl4 (antlr4-python3-runtime; tested with version 4.7.2)
    • Numpy (tested with version 1.16.2)
    • Scikit-learn (tested with version 0.20.3)
  2. Linux packages:
    • gcc (tested with version 7.3.0)
    • make (tested with version 4.1)

Usage

SeeDot can be invoked using SeeDot.py file. The arguments for the script are supplied as follows:

usage: SeeDot.py [-h] [-a] --train  --test  --model  [--tempdir] [-o]

optional arguments:
  -h, --help      show this help message and exit
  -a , --algo     Algorithm to run ('bonsai' or 'protonn')
  --train         Training set file
  --test          Testing set file
  --model         Directory containing trained model (output from
                  Bonsai/ProtoNN trainer)
  --tempdir       Scratch directory for intermediate files
  -o , --outdir   Directory to output the generated Arduino sketch

An example invocation is as follows:

python SeeDot.py -a bonsai --train path/to/train.npy --test path/to/test.npy --model path/to/Bonsai/model

SeeDot expects the train and the test data files in a specific format. Each data file should be of the shape [numberOfDataPoints, numberOfFeatures + 1], where the class label is in the first column. The tool currently support the following file formats for the data files: numpy arrays (.npy), tab-separated values (.tsv), comma-separated values (.csv), and libsvm (.txt).

The path to the trained Bonsai/ProtoNN model is specified in the --model argument. After training, the learned parameters are stored in this directory in a specific format. For Bonsai, the learned parameters are Z, W, V, T, Sigma, Mean, and Std. For ProtoNN, the learned parameters are W, B, and Z. These parameters can be either numpy arrays (.npy) or plaintext files.

The tempdir directory is used to store the intermediate files generated by the compiler. The device-specific fixed-point code is stored in the outdir directory.

Getting started: Quantizing ProtoNN on usps10

To help get started with SeeDot, we provide 1) a pre-loaded fixed-point model, and 2) instructions to generate fixed-point code for the ProtoNN predictor on the usps10 dataset. The process for generating fixed-point code for the Bonsai predictor is similar.

Pre-loaded model

To make it easy to test the SeeDot-generated code, a ready-to-upload Arduino sketch is provided that can be run on an Arduino device without any changes. The sketch is located at Tools/SeeDot/seedot/arduino/arduino.ino and contains pre-loaded ProtoNN model on the usps10 dataset. To upload the sketch to the device, skip steps 1-3 in the below guide and follow the step 4: Prediction on the device.

Generating fixed-point code

This process consists of four steps: 1) installing EdgeML TensorFlow library, 2) training ProtoNN on usps10, 3) quantizing the trained model with SeeDot, and 4) performing prediction on the device.

Step 1: Installing EdgeML TensorFlow library

  1. Clone the EdgeML repository and navigate to the right directory.

    git clone https://github.com/Microsoft/EdgeML
    cd EdgeML/tf/
    
  2. Install the EdgeML library.

    pip install -r requirements-cpu.txt
    pip install -e .
    

Step 2: Training ProtoNN on usps10

  1. Navigate to the ProtoNN examples directory.

    cd examples/ProtoNN
    
  2. Fetch usps10 data and create output directory.

    python fetch_usps.py
    python process_usps.py
    mkdir usps10/output
    
  3. Invoke ProtoNN trainer using the following command.

    python protoNN_example.py --data-dir ./usps10 --projection-dim 25 --num-prototypes 55 --epochs 100 -sW 0.3 -o usps10/output
    

This would give around 90.035% classification accuracy. The trained model is stored in the output directory.

More information on using the ProtoNN trainer can be found here.

Step 3: Quantizing with SeeDot

  1. Navigate to the SeeDot directory and create the output directory.

    cd ../../../Tools/SeeDot
    mkdir arduino
    
  2. Invoke SeeDot using the following command.

    python SeeDot.py -a protonn --train ../../tf/examples/ProtoNN/usps10/train.npy --test ../../tf/examples/ProtoNN/usps10/test.npy --model ../../tf/examples/ProtoNN/usps10/output -o arduino
    

    The SeeDot-generated code would give around 89.985% classification accuracy. The difference in classification accuracy is 0.05% compared to the floating-point code. The generated code is stored in the arduino folder which contains the sketch along with two files: model.h and predict.cpp. model.h contains the quantized model and predict.cpp contains the inference code.

Step 4: Prediction on the device

Follow the below steps to perform prediction on the device, where the SeeDot-generated code is run on a single data-point stored on the device's flash memory.

  1. Open the Arduino sketch file located at arduino/arduino.ino in the Arduino IDE.
  2. Connect the Arduino microcontroller to the computer and choose the correct board configuration.
  3. Upload the sketch to the device.
  4. Open the Serial Monitor and select baud rate specified in the sketch (default is 115200) to monitor the output.
  5. The average prediction time is computed every 100 iterations. On an Arduino Uno, the average prediction time is 35991 micro seconds.

More device-specific details on extending the Arduino sketch for other use cases can be found in arduino/README.md.

The above workflow has been tested on Arduino Uno and Arduino MKR1000. It is expected to work on other Arduino devices as well.

Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT license.

You can’t perform that action at this time.