Sorei

A lightweight CUDA-accelerated neural network training library for C++20. Sorei provides a fluent graph-builder API for defining computation graphs with automatic differentiation.

Requirements

• CMake 3.18+
• C++20-capable compiler
• CUDA Toolkit (cuBLAS)

Building

Sorei is a static library used via CMake's add_subdirectory. To integrate it into a project:

add_subdirectory(path/to/sorei sorei)
target_link_libraries(my_target PRIVATE sorei)

To build standalone:

cmake -B build/release -DCMAKE_BUILD_TYPE=Release
cmake --build build/release -j

Usage

Define a model by subclassing sorei::nn::Model and implementing build_graph:

#include "sorei/nn.h"

struct MyModel : public sorei::nn::Model {
    sorei::nn::GraphOutput build_graph(sorei::nn::GraphBuilder& b) override {
        auto x      = b.input_float("x", {INPUT_DIM, 0});
        auto labels = b.input_int("labels", {1, 0});

        auto l1 = b.affine_layer(INPUT_DIM, HIDDEN_DIM);
        auto l2 = b.affine_layer(HIDDEN_DIM, NUM_CLASSES);

        auto out  = l2(l1(x).relu());
        auto loss = out.softmax_cross_entropy(labels).mean();

        return {.prediction = out, .loss = loss};
    }
};

Then train with an optimizer and a learning-rate scheduler:

MyModel model;
auto optim    = sorei::nn::AdamW(model.params(), 0.9f, 0.999f, 0.01f);
auto lr_sched = sorei::nn::CosineAnnealingLR(lr, lr * 0.1f, epochs);

for (int epoch = 0; epoch < epochs; ++epoch) {
    model.forward({{"x", inputs}, {"labels", targets}});
    model.backward();
    optim.step(lr_sched.get());
    lr_sched.step();
}

Examples

Example	Description
examples/mnist	MLP trained on MNIST handwritten digits
examples/astra	NNUE for my chess engine Astra

Running an Example

Build and run one of the examples:

cd examples/mnist
cmake -B build/release -DCMAKE_BUILD_TYPE=Release
cmake --build build/release -j
./build/release/mnist_example

License

MIT — see LICENSE.