SeeDot is an automatic quantization tool that generates efficient machine learning (ML) inference code for IoT devices.
ML models are usually expressed in floating-point, and IoT devices typically lack hardware support for floating-point arithmetic. Hence, running such ML models on IoT devices involves simulating floating-point arithmetic in software, which is very inefficient. SeeDot addresses this issue by generating fixed-point code with only integer operations. To enable this, SeeDot takes as input trained floating-point models (like Bonsai or ProtoNN) and generates efficient fixed-point code that can run on microcontrollers. The SeeDot compiler uses novel compilation techniques like automatically inferring certain parameters used in the fixed-point code, optimized exponentiation computation, etc. With these techniques, the generated fixed-point code has comparable classification accuracy and performs significantly faster than the floating-point code.
To know more about SeeDot, please refer to our publication here.
This document describes the tool usage with an example.
- Python 3 with following packages:
- Linux packages:
SeeDot can be invoked using
SeeDot.py file. The arguments for the script are supplied as follows:
usage: SeeDot.py [-h] [-a] --train --test --model [--tempdir] [-o] optional arguments: -h, --help show this help message and exit -a , --algo Algorithm to run ('bonsai' or 'protonn') --train Training set file --test Testing set file --model Directory containing trained model (output from Bonsai/ProtoNN trainer) --tempdir Scratch directory for intermediate files -o , --outdir Directory to output the generated Arduino sketch
An example invocation is as follows:
python SeeDot.py -a bonsai --train path/to/train.npy --test path/to/test.npy --model path/to/Bonsai/model
SeeDot expects the
train and the
test data files in a specific format. Each data file should be of the shape
[numberOfDataPoints, numberOfFeatures + 1], where the class label is in the first column. The tool currently support the following file formats for the data files: numpy arrays (.npy), tab-separated values (.tsv), comma-separated values (.csv), and libsvm (.txt).
The path to the trained Bonsai/ProtoNN model is specified in the
--model argument. After training, the learned parameters are stored in this directory in a specific format. For Bonsai, the learned parameters are
Std. For ProtoNN, the learned parameters are
Z. These parameters can be either numpy arrays (.npy) or plaintext files.
tempdir directory is used to store the intermediate files generated by the compiler. The device-specific fixed-point code is stored in the
Getting started: Quantizing ProtoNN on usps10
To help get started with SeeDot, we provide 1) a pre-loaded fixed-point model, and 2) instructions to generate fixed-point code for the ProtoNN predictor on the usps10 dataset. The process for generating fixed-point code for the Bonsai predictor is similar.
To make it easy to test the SeeDot-generated code, a ready-to-upload Arduino sketch is provided that can be run on an Arduino device without any changes. The sketch is located at
Tools/SeeDot/seedot/arduino/arduino.ino and contains pre-loaded ProtoNN model on the usps10 dataset. To upload the sketch to the device, skip steps 1-3 in the below guide and follow the step 4: Prediction on the device.
Generating fixed-point code
This process consists of four steps: 1) installing EdgeML TensorFlow library, 2) training ProtoNN on usps10, 3) quantizing the trained model with SeeDot, and 4) performing prediction on the device.
Step 1: Installing EdgeML TensorFlow library
Clone the EdgeML repository and navigate to the right directory.
git clone https://github.com/Microsoft/EdgeML cd EdgeML/tf/
Install the EdgeML library.
pip install -r requirements-cpu.txt pip install -e .
Step 2: Training ProtoNN on usps10
Navigate to the ProtoNN examples directory.
Fetch usps10 data and create output directory.
python fetch_usps.py python process_usps.py mkdir usps10/output
Invoke ProtoNN trainer using the following command.
python protoNN_example.py --data-dir ./usps10 --projection-dim 25 --num-prototypes 55 --epochs 100 -sW 0.3 -o usps10/output
This would give around 90.035% classification accuracy. The trained model is stored in the
More information on using the ProtoNN trainer can be found here.
Step 3: Quantizing with SeeDot
Navigate to the SeeDot directory and create the output directory.
cd ../../../Tools/SeeDot mkdir arduino
Invoke SeeDot using the following command.
python SeeDot.py -a protonn --train ../../tf/examples/ProtoNN/usps10/train.npy --test ../../tf/examples/ProtoNN/usps10/test.npy --model ../../tf/examples/ProtoNN/usps10/output -o arduino
The SeeDot-generated code would give around 89.985% classification accuracy. The difference in classification accuracy is 0.05% compared to the floating-point code. The generated code is stored in the
arduinofolder which contains the sketch along with two files: model.h and predict.cpp.
model.hcontains the quantized model and
predict.cppcontains the inference code.
Step 4: Prediction on the device
Follow the below steps to perform prediction on the device, where the SeeDot-generated code is run on a single data-point stored on the device's flash memory.
- Open the Arduino sketch file located at
arduino/arduino.inoin the Arduino IDE.
- Connect the Arduino microcontroller to the computer and choose the correct board configuration.
- Upload the sketch to the device.
- Open the Serial Monitor and select baud rate specified in the sketch (default is 115200) to monitor the output.
- The average prediction time is computed every 100 iterations. On an Arduino Uno, the average prediction time is 35991 micro seconds.
More device-specific details on extending the Arduino sketch for other use cases can be found in
The above workflow has been tested on Arduino Uno and Arduino MKR1000. It is expected to work on other Arduino devices as well.
Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT license.