Skip to content

NeuroBrix 0.6.3

Choose a tag to compare

@benkelaya benkelaya released this 09 Nov 15:24
· 91 commits to main since this release

NeuroBrix 0.6.3

NeuroBrix 0.6.3 serves models: neurobrix serve keeps a model warm in a daemon, neurobrix chat talks to it, and the
same daemon now answers an OpenAI-compatible HTTP API (serve --http, default 127.0.0.1:8765: /v1/models,
/v1/chat/completions with streaming and tool calls, /v1/completions, /v1/images/generations,
/v1/audio/transcriptions, /v1/audio/speech). A conversation keeps its KV cache between turns, so each turn prefills
only its new tokens (run --turns does the same from the command line). Every model has one name, its hub name
(TinyLlama/TinyLlama-1.1B-Chat-v1.0), recorded at import.

The terminal shows what the user needs (-v / -vv for the rest), and the user-path failures of 0.6.2 are fixed or
said in one line. Laptops with an integrated GPU are detected from what the operating system ships: an integrated GPU
is planned as unified memory, and a GPU no backend drives is refused by name while the machine runs on its CPU. On
Apple GPUs, a first run that meets an out-of-memory now re-plans and continues. The macOS 14 Metal wheels (Python 3.10–3.13) are published and install by hand (triton-ext release); install-metal installs macOS 15 / Python 3.12 only, and chooses the macOS 14 wheels from nbxtensor 0.0.9 / engine 0.6.4.

Install: pip install neurobrix (the Triton engine, no PyTorch) or pip install "neurobrix[pytorch]"; on a Mac,
python -m nbxtensor install-metal. The library is nbxtensor 0.0.8.

The full list, the fixes and the known limits of this release are in the CHANGELOG.