Repository navigation
NeuroBrix 0.6.15
NeuroBrix 0.6.15
With nbxtensor 0.1.7: Kokoro-82M speaks with its own voice on the Triton engine, neurobrix upscale runs on every host as
neurobrix run does, neurobrix import installs only the container the hub states, and large models run and serve within
the memory their plan priced.
Changed
- A model larger than its card runs on the Triton engine within its plan. When a model's layers stream from host
memory, the weights loaded on the card from the start (embeddings, norms) count against the layers the card keeps:
Qwen2.5-Coder-14B runs on one 16 GB V100. - Long served prompts run on the PyTorch engine with the memory their plan priced. The check that a model's attention
mask is the causal pattern runs once per prefill chunk, reading the mask in bands no larger than the mask itself:
Qwen3-30B-A3B-Thinking and Qwen3-Coder-30B-A3B serve every request with--compiledon one 32 GB V100. - Kokoro-82M speaks with its own voice on the Triton engine. A model built in float32 and run in half precision keeps
float32 wherever its own forward keeps it, unless the model's calibration record measured that op safe in half: the
voice's harmonic source (a running phase of thousands of radians) is computed whole, and the Triton engine's voice
matches the vendor's reference render and the PyTorch engine's. neurobrix importinstalls only the container the hub states. The downloaded.nbxis checked against the
sha256 the hub publishes for it, whatever host served the bytes; another archive is refused by name with both
values, kept beside the store as<name>.nbx.unverifiedand never extracted.neurobrix upscalechooses its engine asneurobrix rundoes, on every host: the same--compiled,
--sequential,--triton,--triton-sequentialflags, the PyTorch engine where it is installed and the Triton engine
where it is not when no flag is given, said on the launch line. On a host without a GPU,upscaleruns an upscaler on
the PyTorch engine asrun --input-imagedoes.--modestays accepted as the same choice.- A language model split across two cards scores every sequence of a batch. When the last layer and the output head
sit on different cards, each sequence's last position reaches the head through the tensor library's card-to-card
transfer, laid out per sequence.
nbxtensor 0.1.7: https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md