Neural Network library for embedded systems

Copyright (c) 2019-2026 SynthInt Technologies, LLC

https://synthint.ai

SPDX-License-Identifier: Apache-2.0

This is a lightweight neural network library for use in microcontrollers and embedded systems.

The code is divided into the following sections:

1. nn.[ch] - The neural net library, which can be pulled directly into your embedded project.

2. data_prep.[ch] - Data processing functions, used to read, parse, and shuffle training data.

3. dequantize.c - Converts an 8-bit integer model into a floating point model.

4. train.c - An example of how to construct, train, and save a neural network model.

5. test.c - Evaluates model performance, comparing predictions to ground truth of seen vs. unseen data.

6. predict.c - Demonstrates how to use a trained neural network model in a target application to make predictions on new data.

7. prune.c - Removes least contributing neurons from a network to reduce model size and improve performance.

8. quantize.c - Converts a floating-point model to a 8-bit integer model.

9. summary.c - Describes a model file, including which of the three formats (ASCII, binary, or inplace) it's saved in.

With this library, neural networks of any width and depth may be constructed and trained. The following activation functions are supported:

• Identity
• Linear
• ReLU
• Leaky ReLU
• ELU
• Threshold
• Sigmoid
• TanH
• GELU
• SiLU
• Softmax (output layer only -- paired automatically with cross-entropy loss; see ACTIVATION_FUNCTION_TYPE_SOFTMAX in nn.h)

Different activation functions may be assigned to each layer in the network.

A bias can be added to each layer independently.

The following layer types are supported (added one at a time, in order, via nn_add_layer()):

• Input - holds the network's input values; has no weights, bias, or activation of its own.
• Fully Connected (FC) - every neuron connects to every neuron in the previous layer; the standard building block for dense layers.
• Convolutional (CNN) - 2D convolution over the previous layer's feature maps, with configurable kernel size, stride, padding (same/zero-padding), and channel counts (see cnn_t in nn.h).
• Pooling - downsamples a CNN layer's feature maps; Min, Max, or Average (see pooling_type_t in nn.h).
• Dropout - training-only regularization that randomly zeroes a configurable fraction of a layer's outputs each step; a no-op pass-through at inference (see dropout_t in nn.h).
• Output - the network's final layer; computed the same way as Fully Connected, with support for Softmax (see Features above) in addition to the other activation functions.

LSTM, GRU, RNN, Attention, and Transformer layer types are declared in layer_type_t but not yet implemented (see TODO below).

To build the nn library and sample training and prediction programs, just type:

make

To train:

./train model.txt

The model can be further trained (or fine-tuned) simply by re-running the training program, which further trains a model file if it already exists.

The included example data is the MNIST data set.

To evaluate the model performance:

./test model.txt

To use the trained model:

./predict

To prune the model (this example removes the 10 least contributing neurons):

./prune model.txt 10

To quantize the trained model (which is floating point by default), run the following command:

./quantize model.txt model_quantized.txt

To export a trained model as a flash-resident binary for a microcontroller target, add --inplace so export writes the zero-copy "inplace" format instead of the regular binary format. export accepts any of the three model formats as input (it auto-detects ASCII, binary, or an existing inplace file), so this works directly on whichever one you have:

./export model.txt model_inplace.bin --inplace

See Embedding an inplace model as a C header below for turning that file into a .h you can #include and pass to nn_load_model_inplace(). Omit --inplace to instead get the regular binary format (nn_load_model_binary()/nn_load_model_memory()).

To check which format a model file is in (along with its version, quantization status, and layer-by-layer layout), run:

./summary model.txt

The network architecture is a fully connected feed-forward neural network. It is based on floating-point computation. The widths of each layer, the activation function to be used, and the bias for each layer is set as each successive layer is added to the network using the nn_add_layer() function call. Multiple layers may be added to construct a deep neural network.

This embedded neural network library was used to power a handwritten character recognition application running on an STM32H7 microcontroller. Check it out here:

https://www.youtube.com/watch?v=cqjwSkrGtww

The model can be saved in the following formats:

• ASCII - a text file of floating-point values. The first line depicts the number of layers, inclusive of the input and output layers. The construct of each of those layers comprises the next set of lines, one line for each layer. The format of each line is width (in neurons), activation function, and bias. The remaining lines are the weights of each neuron in each layer, for all layers. Since there are no weights associated with the neurons in the input layer, these are skipped, and do not exist in the model file.

• Binary - a compact, raw binary encoding of the same information. Every binary model file begins with the 4-byte magic number NNB1, followed by the same fields the ASCII format stores (quantized flag, version, layer definitions, weights, and biases), written as raw integers/floats rather than text.

nn_load_model() reads a model file's first few bytes and dispatches to the ASCII or binary loader automatically based on the magic number, so any tool that calls it can open either kind of model file without knowing in advance which format it's in. nn_save_model() writes binary format when the destination path ends in .bin (case-insensitive) and ASCII format otherwise. train, test, predict, prune, quantize, dequantize, and summary all use these, so passing e.g. model.bin instead of model.txt is enough to train, evaluate, prune, (de)quantize, or run inference against a binary model file. nn_load_model_ascii/nn_save_model_ascii and nn_load_model_binary/nn_save_model_binary remain available for callers that need to force a specific format regardless of extension (as import does, to convert binary back to ASCII).

nn_model_format(path) peeks a file's first few bytes (without loading it) to report which of the three formats -- ASCII, binary, or inplace -- it's in; summary uses this to print a Model Format: line, and export uses it to accept any of the three as input (reading an inplace file into a buffer and loading it with nn_load_model_inplace(), since that format has to be read from memory rather than a plain path) regardless of which one it writes as output. export also accepts an --inplace flag to write the inplace format (below) instead of the regular binary format.

For targets with no filesystem, nn_load_model_memory(data, size) parses that same binary format directly out of a caller-supplied buffer (e.g. a model baked into flash as a byte array on a microcontroller) instead of reading from a file, with no FILE*/fopen dependency. It copies the model into its own allocations, so data only needs to stay valid for the duration of the call.

• Inplace - a third format (magic NNP1), written by nn_save_model_inplace() and read back with zero copy by nn_load_model_inplace(data, size): instead of parsing weights/biases into freshly malloc'd RAM, the returned model's weight (or weight_quantized) and bias (or bias_quantized) arrays point directly into the caller's buffer. This is the format for microcontroller targets where RAM, not flash, is the tight resource -- the buffer (typically a static const uint8_t[] baked into flash) must stay valid for as long as the model is used, and the model it produces is read-only: nn_train(), nn_quantize(), nn_dequantize(), nn_remove_neuron(), and nn_prune_lightest_neuron() all refuse to run against it. Use nn_predict()/nn_error() for inference as usual, and release it with nn_free() as usual -- it knows not to free the aliased buffers.

To use this nn library in your own embedded system, it is only necessary to pull in the nn.c and nn.h files into your project. The other source files in the nn package are intended for data preparation for offline training, as well as examples of training and inference. For microcontroller-class inference-only use, nn_load_model_inplace() (see Model File Format above) is the intended entry point: it runs inference directly out of a flash-resident model with no RAM copy of the weights; nn_load_model_memory() is the fallback when the model instead needs to be copied into (and is free to be modified/replaced in) RAM.

To integrate an inplace-format model into firmware as a flash-resident array, convert the file nn_save_model_inplace() wrote with xxd -i:

xxd -i model_inplace.bin > model_inplace.h

This generates a .h file with a unsigned char model_inplace_bin[] = {...}; array and a unsigned int model_inplace_bin_len byte count. #include it, then hand the array straight to nn_load_model_inplace():

#include "model_inplace.h"

nn_t *nn = nn_load_model_inplace(model_inplace_bin, model_inplace_bin_len);

xxd -i doesn't add an alignment attribute, so on a toolchain/linker that doesn't already place .rodata arrays 4-byte aligned, add one by hand to the generated declaration (unsigned char model_inplace_bin[] __attribute__((aligned(4))) = {...};) -- nn_load_model_inplace() requires 4-byte alignment (see its comment in nn.c).

Copyright (c) 2019-2026 SynthInt Technologies, LLC. All rights reserved.

Licensed under the Apache License 2.0.

• Add Recurrent Neural Network Layer (RNN) layer type
• Add Long Short-Term Memory (LSTM) layer type
• Add Gated Recurrent Unit (GRU) layer type
• Add Attention layer type
• Add Transformer layer type