Repository navigation
Releases
Release v0.1.1
Released 2026-11-16. A patch release. It rewords one comment in the public interface and one contributor rule. The library's code and behavior are the same as v0.1.0, which lists everything the library holds.
What it includes
| Change | What |
|---|---|
| #8 | The comment on model::OnToken (include/model/engine.hpp) now names the token callback as an injected dependency, without an internal decision number. The matching naming rule in CLAUDE.md is reworded. |
| this release | The version 0.1.1 (project(VERSION) in CMakeLists.txt and model::kVersion), held by the tests, and these notes. The v0.1.0 notes are rewrapped so their release page renders without a stray line break. |
What it does not include
- No code change: the interface, the stub engine and the MLX backend are those of v0.1.0.
- No libtorch, llama.cpp or CUDA backend yet. Their CMake options exist, but each returns the stub engine.
- No server and no image. Issue #3 tracks serving a model over an OpenAI-compatible HTTP API, as an image.
- No pull request was open at the cut, so nothing was deferred.
Known issues
- None filed. As in v0.1.0, the test that round-trips text through a real tokenizer model runs only when
ORAMA_TOKENIZER_MODELnames a SentencePiece model; without one it passes without checking anything, and it did not run for this release.
Verification
| Proof | Result |
|---|---|
Default build (no backend), ctest, macOS arm64 |
11 tests, all pass |
MLX build (-DORAMA_BACKEND_MLX=ON, mlx 0.31.2), ctest, macOS arm64 |
18 tests, all pass |
tools/spdx-headers.py --check |
passes |
tools/sbom.py --check |
passes, regenerated after the last change |
Not run: the real-tokenizer round trip and a Gemma-2 generation (no model weights on the release machine), and any Linux build.
Release v0.1.0
Released 2026-11-16. The first release. A C++ library for training and running a model behind one interface, with a native backend per platform. This release has the interface, a deterministic stub engine that runs everywhere, and the MLX backend for Apple Silicon, which loads and runs a real Gemma-2 model with no Python. Because it is the first release, this page lists everything in it.
What it includes
The library: orama::model-runtime
model::Engine(include/model/engine.hpp), one interface for every backend:infer(prompt, on_token): a token-streaming decode. Theon_tokencallback receives each token and may return text to add to the context before the next token, so a caller can fold search or tool results into the decode as it runs.train_step(samples): one LoRA-style training step over a few(prompt, target)pairs, returning the loss.name(),backend(),is_stub().
model::make_engine(backend)builds the engine for a backend;model::default_backend()picks the platform's native one (MLX on macOS, libtorch elsewhere).model::Backend:MLX,LibTorch,LlamaCpp,CUDA, each withlabel().- The stub engine: deterministic, streaming and steerable, with a training loss that falls step by step. It is what every backend without a real implementation returns, so the library builds and its callers run with no machine-learning toolchain.
- Errors are values: methods return
model::Result<T>(include/model/error.hpp), and no exception crosses the interface. model::kNameandmodel::kVersion(include/model/version.hpp) name the library and its version,0.1.0.
The MLX backend (-DORAMA_BACKEND_MLX=ON, macOS on Apple Silicon)
- A real MLX engine on Metal: an on-device LoRA training step that lowers the loss, and inference that still streams and accepts injected context.
- Gemma-2 in C++ (
include/model/gemma.hpp): loadsmlx-community4-bit weights and generates, with a cumulative repetition penalty over the generated tokens and a cycle backstop against greedy loops. - A SentencePiece tokenizer (
model::Tokenizer) and a safetensors loader (model::Weights), both in C++. gemma-cli <model-dir> [prompt]: loads a Gemma-2 model and generates.gemma-bench: a performance harness, run withsudo, that reports prefill and decode tokens per second, GPU residency, power and frequency, and peak memory.GEMMA_STATSprints the same speeds from any run.- gemma-runbook.md: downloading the weights, building, running, checking the output and benchmarking.
Build
- C++23, CMake 3.28 or later with Ninja, Catch2 fetched with FetchContent.
- Backends are CMake options, all off by default:
ORAMA_BACKEND_MLX,ORAMA_BACKEND_LIBTORCH,ORAMA_BACKEND_LLAMACPP,ORAMA_BACKEND_CUDA. - The version lives in
project(VERSION)inCMakeLists.txt; a test checks the header says the same, and two more check this page exists and is indexed.
Licence and project files
- MIT
LICENSEandCOPYRIGHT, anSPDX-License-Identifier: MITline in every source file,THIRD_PARTY_LICENSES(generated bytools/third-party-licenses.py),CONTRIBUTING.md,CLA.mdandCODE_OF_CONDUCT.md(#5). SECURITY.md: how to report a vulnerability privately (#2).- The public tree carries nothing internal (#4).
- A software bill of materials,
sbom.spdx.jsonandsbom.cdx.json, regenerated bytools/sbom.py.
Changes
| Change | What |
|---|---|
| scaffold | the C++23 library, CMake, FetchContent and Catch2 |
| engine | the Engine interface and the deterministic stub backend |
| MLX | the MLX backend: on-device LoRA training step and inference |
| Gemma-2 | the tokenizer and safetensors loader, then Gemma-2 4-bit generation, its fixes for greedy loops, GEMMA_STATS, gemma-bench, and the runbook |
| #2 | SECURITY.md |
| #4 | the tree prepared for the public export |
| #5 | MIT licence, SPDX headers, third-party notices and contributor documents |
| this release | the version 0.1.0, held by tests, and these notes |
What it does not include
- No libtorch, llama.cpp or CUDA backend yet. Their CMake options exist, but each returns the stub engine. On Linux and Windows,
make_engine()gives the stub. - No server and no image. Serving a model over an OpenAI-compatible HTTP API, as an image, is #3.
- No pull request was open at the cut, so nothing was deferred.
Known issues
- None filed. The test that round-trips text through a real tokenizer model runs only when
ORAMA_TOKENIZER_MODELnames a SentencePiece model; without one it passes without checking anything, and it did not run for this release (below).
Verification
| Proof | Result |
|---|---|
Default build (no backend), ctest, macOS arm64 |
11 tests, all pass |
MLX build (-DORAMA_BACKEND_MLX=ON, mlx 0.31.2), ctest, macOS arm64 |
18 tests, all pass |
tools/spdx-headers.py --check |
passes |
tools/sbom.py --check |
passes, regenerated after the last change |
Not run: the real-tokenizer round trip and a Gemma-2 generation (no model weights on the release machine), and any Linux build.