Repository navigation
NeuroBrix 0.6.16
With nbxtensor 0.1.8: a language model served on one NVIDIA card replays its decode as CUDA graphs by default, the
video models that decode their frames in a stream run under the Triton engine again, and neurobrix run --profile-kernels shows where the GPU time goes on Apple GPUs.
Added
neurobrix run --profile-kernelsshows where the GPU time goes on Apple GPUs. The run prints its kernels ranked
by GPU time (and the copies as one row);--profile-kernels PATHalso writes the whole table as JSON. Each kernel is
timed on its own, so the run is slower while it is measured; the answer is the same.
Changed
- A language model served on one NVIDIA card replays its decode as CUDA graphs by default. On a container
certified for NVIDIA cards the answer is byte-identical, and no setting is needed to turn it on. - A served conversation on Apple Silicon plans its prompt in chunks priced on the chip's own tables, and a Mac's
single memory pool is planned as the one cache it is, as on other hardware.
Fixed
- The video models that decode their frames in a stream (Wan2.1, Mochi) run under the Triton engine on a GPU,
where they crashed when the first stored frame state was released. - A model whose
tokenizer.jsonsplits text with Unicode categories, possessive quantifiers or line anchors
tokenizes as its vendor does; a property the engine cannot read is refused by name, never split another way. - A chat template that marks the assistant's turns with
{% generation %}renders as its vendor renders it, where
it stopped at that tag before. - A GPU that no hardware profile describes is refused by name at once, naming the device it could not place.
- A hub that does not answer is said by name.
neurobrix importandneurobrix hubprint the hub's address, why
it did not answer and what to do, then exit 1: no traceback, and no download tried from another source.
nbxtensor 0.1.8: https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md