- Updated uTensor ReadMe
- Updated uTensor-CLI ReadMe
- ROM Tensor support
uTensor is an extremely light-weight machine learning inference framework built on Mbed and Tensorflow. It consists of a runtime library and an offline tool. The total size of graph definition and algorithm implementation of a 3-layer MLP produced by uTensor is less than 32kB in the resulting binary (excluding the weights).
A model is constructed and trained in Tensorflow. uTensor takes the model and produces a .cpp and .hpp file. These files contains the generated C++11 code needed for inferencing. Working with uTensor on the embedded side is as easy as copy-and-paste. The function interface looks like this:
#include "models/deep_mlp.hpp"
...
Context ctx; //creating a context
...
//preparing for the input tensor
...
get_deep_mlp_ctx(Context& ctx, Tensor* input_0); //perform inference
ctx.eval();
S_TENSOR prediction = ctx.get({"y_pred:0"}); //getting the result
The .hpp and .cpp files can be generated given a model (protocal buffer) file, for example:
$ utensor-cli deep_mlp.pb --output-nodes=y_pred
...
... Generate weight file: models/deep_mlp_weight.hpp
... Generate header file: models/deep_mlp.hpp
... Generate source file: models/deep_mlp.cpp
The project is work-in-progress. Here are the operators, of their quantized versions, that are currently avaliable:
- Add
- ArgMax
- Dropout
- MatMal
- Max
- Min
- Placeholder
- Quantization Ops
- ReLu
- Reshape
uTensor should support any Mbed enabled board that has sufficient memory (128+ kB RAM and 512kB+ flash recommended). However, these two boards are popular among the core developers:
- FRDM-K66F: reference development environment
- DISCO-F413ZH: a good demo/application prototyping platform, wi-fi
- Any Mbed board with sufficient memory
There are two flows to get started with uTensor. For Windows users, please choose the Cloud9 flow as shown below.
- Local Installation
- Requires Mbed-CLI installation
- Requires uTensor-CLI installation
- The Cloud9 Flow
- Requires Amazon Cloud9 Account
- Does not support runtime debugging
Step-by-step instructions on how to create your own uTensor project from scratch. You will need a K66F board a micro-SD card (32GB max) for this guide.
The example uses a 3-layer MLP trained on the MNIST dataset. The touch screen input is fed into the neural network for processing and the result is printed on the screen.
This example shows how to buffer time-series data into batches of snapshots. These snapshots are then fed into the neural network for inferencing. The model a small multi-layer MLP trained on the ADL dataset.
uTensor is young and under going rapid development. Many exciting features are on the way:
- Convolution
- Pooling
- CMSIS-NN integration
- Smaller binary
- More efficient Run-time
You can also check the project page for the latest progress. If you'd like to take part in this project, please have a look at our contributor guide and feel free to reach out to us.