Skip to content

NeuroBrix 0.6.16

Latest

Choose a tag to compare

@benkelaya benkelaya released this 15 Nov 10:21
· 37 commits to main since this release

NeuroBrix 0.6.16

With nbxtensor 0.1.8: a language model served on one NVIDIA card replays its decode as CUDA graphs by default, the
video models that decode their frames in a stream run under the Triton engine again, and neurobrix run --profile-kernels shows where the GPU time goes on Apple GPUs.

Added

  • neurobrix run --profile-kernels shows where the GPU time goes on Apple GPUs. The run prints its kernels ranked
    by GPU time (and the copies as one row); --profile-kernels PATH also writes the whole table as JSON. Each kernel is
    timed on its own, so the run is slower while it is measured; the answer is the same.

Changed

  • A language model served on one NVIDIA card replays its decode as CUDA graphs by default. On a container
    certified for NVIDIA cards the answer is byte-identical, and no setting is needed to turn it on.
  • A served conversation on Apple Silicon plans its prompt in chunks priced on the chip's own tables, and a Mac's
    single memory pool is planned as the one cache it is, as on other hardware.

Fixed

  • The video models that decode their frames in a stream (Wan2.1, Mochi) run under the Triton engine on a GPU,
    where they crashed when the first stored frame state was released.
  • A model whose tokenizer.json splits text with Unicode categories, possessive quantifiers or line anchors
    tokenizes as its vendor does
    ; a property the engine cannot read is refused by name, never split another way.
  • A chat template that marks the assistant's turns with {% generation %} renders as its vendor renders it, where
    it stopped at that tag before.
  • A GPU that no hardware profile describes is refused by name at once, naming the device it could not place.
  • A hub that does not answer is said by name. neurobrix import and neurobrix hub print the hub's address, why
    it did not answer and what to do, then exit 1: no traceback, and no download tried from another source.

nbxtensor 0.1.8: https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md