TinyTensor is an efficient lightweight deep learning inference framework.

About

TinyTensor supports a variety of popular neural network architectures such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected neural networks, and it can be used for tasks such as image classification, object detection, speech recognition, and natural language processing.

Development Environment

• Development language: C++ 20
• Math Library: Armadillo
• Logging framework：Google glog
• Unit test: Google Test
• Code style: Clang format
• Performance testing： Benckmark

How to build on Linux

Ubuntu 18 (Debian 10)

apt update
apt install cmake libopenblas-dev liblapack-dev \
libarpack2-dev libsuperlu-dev libomp-dev libopencv-dev

Install Armadillo

wget https://sourceforge.net/projects/arma/files/armadillo-12.2.0.tar.xz
mkdir build && cd build
cmake ..
make -j8
make install

Install Benchmark

cd third_party
git submodule update --init
mv googletest benchmark
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE ../benchmark
make -j8
# 如果想全局安装就接着运行下面的命令
sudo make install

Operators Currently Implemented

• ReLU
• Sigmoid
• Conv
• MaxPooling

Performance Testing

Test Equipment

Intel(R) Xeon(R) W-2223 CPU @ 3.60GHz

Compilation Environment

gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

Performance Results

Input size	Model	Computing Device	Time
224×224 batch = 8	ResNet18	CPU(armadillo)	55ms / image
224×224 batch =16	ResNet18	CPU(armadillo)	28.5ms / image

Acknowledgement

caffe