Repository navigation
Release v0.1.0
Released 2026-11-16. The first release. A C++ library for training and running a model behind one interface, with a native backend per platform. This release has the interface, a deterministic stub engine that runs everywhere, and the MLX backend for Apple Silicon, which loads and runs a real Gemma-2 model with no Python. Because it is the first release, this page lists everything in it.
What it includes
The library: orama::model-runtime
model::Engine(include/model/engine.hpp), one interface for every backend:infer(prompt, on_token): a token-streaming decode. Theon_tokencallback receives each token and may return text to add to the context before the next token, so a caller can fold search or tool results into the decode as it runs.train_step(samples): one LoRA-style training step over a few(prompt, target)pairs, returning the loss.name(),backend(),is_stub().
model::make_engine(backend)builds the engine for a backend;model::default_backend()picks the platform's native one (MLX on macOS, libtorch elsewhere).model::Backend:MLX,LibTorch,LlamaCpp,CUDA, each withlabel().- The stub engine: deterministic, streaming and steerable, with a training loss that falls step by step. It is what every backend without a real implementation returns, so the library builds and its callers run with no machine-learning toolchain.
- Errors are values: methods return
model::Result<T>(include/model/error.hpp), and no exception crosses the interface. model::kNameandmodel::kVersion(include/model/version.hpp) name the library and its version,0.1.0.
The MLX backend (-DORAMA_BACKEND_MLX=ON, macOS on Apple Silicon)
- A real MLX engine on Metal: an on-device LoRA training step that lowers the loss, and inference that still streams and accepts injected context.
- Gemma-2 in C++ (
include/model/gemma.hpp): loadsmlx-community4-bit weights and generates, with a cumulative repetition penalty over the generated tokens and a cycle backstop against greedy loops. - A SentencePiece tokenizer (
model::Tokenizer) and a safetensors loader (model::Weights), both in C++. gemma-cli <model-dir> [prompt]: loads a Gemma-2 model and generates.gemma-bench: a performance harness, run withsudo, that reports prefill and decode tokens per second, GPU residency, power and frequency, and peak memory.GEMMA_STATSprints the same speeds from any run.- gemma-runbook.md: downloading the weights, building, running, checking the output and benchmarking.
Build
- C++23, CMake 3.28 or later with Ninja, Catch2 fetched with FetchContent.
- Backends are CMake options, all off by default:
ORAMA_BACKEND_MLX,ORAMA_BACKEND_LIBTORCH,ORAMA_BACKEND_LLAMACPP,ORAMA_BACKEND_CUDA. - The version lives in
project(VERSION)inCMakeLists.txt; a test checks the header says the same, and two more check this page exists and is indexed.
Licence and project files
- MIT
LICENSEandCOPYRIGHT, anSPDX-License-Identifier: MITline in every source file,THIRD_PARTY_LICENSES(generated bytools/third-party-licenses.py),CONTRIBUTING.md,CLA.mdandCODE_OF_CONDUCT.md(#5). SECURITY.md: how to report a vulnerability privately (#2).- The public tree carries nothing internal (#4).
- A software bill of materials,
sbom.spdx.jsonandsbom.cdx.json, regenerated bytools/sbom.py.
Changes
| Change | What |
|---|---|
| scaffold | the C++23 library, CMake, FetchContent and Catch2 |
| engine | the Engine interface and the deterministic stub backend |
| MLX | the MLX backend: on-device LoRA training step and inference |
| Gemma-2 | the tokenizer and safetensors loader, then Gemma-2 4-bit generation, its fixes for greedy loops, GEMMA_STATS, gemma-bench, and the runbook |
| #2 | SECURITY.md |
| #4 | the tree prepared for the public export |
| #5 | MIT licence, SPDX headers, third-party notices and contributor documents |
| this release | the version 0.1.0, held by tests, and these notes |
What it does not include
- No libtorch, llama.cpp or CUDA backend yet. Their CMake options exist, but each returns the stub engine. On Linux and Windows,
make_engine()gives the stub. - No server and no image. Serving a model over an OpenAI-compatible HTTP API, as an image, is #3.
- No pull request was open at the cut, so nothing was deferred.
Known issues
- None filed. The test that round-trips text through a real tokenizer model runs only when
ORAMA_TOKENIZER_MODELnames a SentencePiece model; without one it passes without checking anything, and it did not run for this release (below).
Verification
| Proof | Result |
|---|---|
Default build (no backend), ctest, macOS arm64 |
11 tests, all pass |
MLX build (-DORAMA_BACKEND_MLX=ON, mlx 0.31.2), ctest, macOS arm64 |
18 tests, all pass |
tools/spdx-headers.py --check |
passes |
tools/sbom.py --check |
passes, regenerated after the last change |
Not run: the real-tokenizer round trip and a Gemma-2 generation (no model weights on the release machine), and any Linux build.