A dependency-free neural network library written from scratch in C — including convolutional networks — that trains a ~99.3%-accurate MNIST digit classifier and beats classical baselines on time-series forecasting, with no libraries, no frameworks, just C99 and the standard math library.
The engine implements the pieces that make a network actually work: dense and convolutional / max-pooling layers, configurable activations (sigmoid / tanh / ReLU + softmax), MSE and cross-entropy losses, SGD-with-momentum and Adam, mini-batch training, principled weight initialisation (Xavier / He), and L2 weight decay. The convolutional backprop is verified against finite-difference gradients, and everything is tested, benchmarked, and CI-checked on Linux, macOS and Windows.
Every number here is produced by code in this repo and checked by CI — no hand-waving.
| Model | Test accuracy |
|---|---|
| CNN + data augmentation (conv8 → pool → conv16 → pool → dense128 → 10) | ~99.3% |
| CNN (no augmentation) | ~99.1% |
MLP (784-128-10, ReLU + softmax, Adam) |
~97.9% |
All pure C99, no dependencies. The MLP trains in ~110 s; the CNN reaches 99.1% in ~10 epochs
and 99.3% with --augment (random pixel shifts), using a learning-rate decay schedule.
scripts/get_mnist.sh # download the dataset (~11 MB)
make mnist && ./build/mnist_train # the MLP
make mnist-cnn && ./build/mnist_cnn --augment 2 # the CNN, with augmentation
./build/mnist_cnn --save digits.cnn # save the trained model
./build/mnist_cnn --load digits.cnn # reload and evaluate, no retrainingCI trains fast subsets of both every push and fails if accuracy drops below its floor, so
these numbers can't silently rot. The CNN's backprop is verified against finite-difference
gradients (cnn_gradient_check) in the test suite.
The same convolutional engine on 3-channel 32x32 colour photos (a much harder problem
than digits) — conv16 → pool → conv32 → pool → dense128 → 10:
| Metric | Value |
|---|---|
| Test accuracy | ~72% (12 epochs, augmentation) |
| Input | 3x32x32 RGB |
scripts/get_cifar10.sh
make cifar-cnn && ./build/cifar_cnn --augment 1Multi-channel convolution is the same code path as MNIST — only the input shape changes.
The same loader and trainer, pointed at a different and harder dataset with zero code changes, reaches ~88.6% — proof the engine isn't overfit to one benchmark:
scripts/get_fashion_mnist.sh
./build/mnist_train --train-images data/fashion/train-images-idx3-ubyte \
--train-labels data/fashion/train-labels-idx1-ubyte \
--test-images data/fashion/t10k-images-idx3-ubyte \
--test-labels data/fashion/t10k-labels-idx1-ubyteForecasting annual sunspot activity from the prior 30 years (evaluation 1955–1979). The network beats both a naive baseline and a linear autoregression:
| Model | Eval RMSE |
|---|---|
| Neural net (open-loop) | 0.129 |
| Linear AR baseline | 0.141 |
| Naive / persistence | 0.198 |
- Layers: fully-connected, plus 2D convolution, max-pooling and flatten (CNNs)
- Activations: sigmoid, tanh, ReLU (hidden) + softmax (classification output)
- Losses: mean squared error (regression) and cross-entropy (classification)
- Optimizers: SGD with momentum, and Adam
- Training: online or mini-batch; early stopping; L2 weight decay; LR decay; dropout; data augmentation
- Models: save / load trained CNNs to disk (
cnn_save/cnn_load) - Init: Xavier/Glorot (sigmoid/tanh), He (ReLU)
- Reproducible: self-contained PCG32 RNG (no global
rand()); fixed seed → fixed result - Portable: C99, builds clean on GCC / Clang / MSVC with
-Wall -Wextra -Wpedantic - Parallel (optional): OpenMP multi-threads the CNN conv/dense layers
(
-DNEURALNET_ENABLE_OPENMP=ON); gradients stay bit-identical to the serial build - Embeddable: the engine never calls
exit(); errors are returned
You need a C99 compiler. Either build system works.
cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure # unit + regression + classification tests
./build/neuralnet # the sunspot forecaster
./build/mnist_train # the MNIST classifier (after get_mnist.sh)Options: -DNEURALNET_WARNINGS_AS_ERRORS=ON, -DNEURALNET_ENABLE_SANITIZERS=ON,
-DNEURALNET_ENABLE_COVERAGE=ON, -DNEURALNET_BUILD_EXAMPLES=OFF.
make # build/neuralnet
make mnist # build/mnist_train
make test # build and run all testsscripts/get_mnist.sh # fetch data into data/mnist/
./build/mnist_train # train (defaults: 128 hidden, Adam, 10 epochs)
./build/mnist_train --hidden 256 --epochs 15 # bigger net, longer
./build/mnist_train --save digits.weights # save the trained model
./build/mnist_train --help./neuralnet # train on the built-in data
./neuralnet --config config/default.cfg # settings from a file
./neuralnet --data my.csv --layers 30,20,1 --activation relu
./neuralnet --predict predictions.csv # write predictions to CSV
./neuralnet --save model.weights # save / --load to reuse
./neuralnet --helpSee ./neuralnet --help for the full flag list (learning rate, momentum, seed, split, etc.).
src/
network.{c,h} engine: layers, activations, losses, optimizers, mini-batch
rng.{c,h} reproducible PCG32 generator
util.{c,h} validated, locale-independent numeric parsing
dataset.{c,h} time-series loading + normalisation
baseline.{c,h} naive + linear-AR reference forecasters
config.{c,h} CLI flags + config-file parsing + validation
forecast.{c,h} training loop, early stopping, evaluation, metrics
cnn.{c,h} convolutional engine: conv/pool/flatten/dense/softmax + Adam
mnist.{c,h} IDX-format MNIST loader
main.c sunspot forecaster CLI
examples/
mnist.c MLP MNIST trainer mnist_cnn.c convolutional MNIST trainer
tests/
test_network.c engine units (forward, activations, XOR, weight I/O, RNG)
test_classify.c softmax + cross-entropy + Adam + mini-batch (4-quadrant task)
test_forecast.c golden regression: the net must beat the naive baseline
test_cnn.c finite-difference gradient check of conv/pool/dense backprop
scripts/
get_mnist.sh download MNIST plot.py matplotlib visualisation
Each non-input neuron computes an activation of the weighted sum of the previous layer's
outputs plus a bias. Training minimises the loss by gradient descent: the output error term
is act'(out)·(target−out) for MSE, or simply (out−target) for softmax + cross-entropy;
hidden errors are back-propagated through the downstream weights. SGD applies
Δw = eta·grad + alpha·Δw_prev; Adam keeps per-weight first/second moment estimates for an
adaptive step. Weights start from an activation-matched scheme (Xavier or He), and inputs are
scaled into the activation's responsive range.
Further reading: https://en.wikipedia.org/wiki/Backpropagation
Browse the generated API reference at https://bikebrainz.github.io/NeuralNetwork/
(built from source with Doxygen on every push to main).
See CONTRIBUTING.md. CI enforces builds (no warnings), tests, sanitizers, Valgrind, clang-tidy, cppcheck, clang-format, coverage, and the MNIST / Fashion-MNIST / CIFAR-10 accuracy floors.
Originally written by Dean Urschel; reworked into a configurable, tested, benchmarked, cross-platform library by Gavin Chase.