Skip to content

NeuroBrix 0.6.0 — the Triton engine on nbxtensor

Choose a tag to compare

@benkelaya benkelaya released this 07 Nov 01:19
· 290 commits to main since this release

The release that moves NeuroBrix's Triton engine onto nbxtensor (0.0.4). Its final gate ran the 49 catalogue models
through the entry point in both engines on both V100 card classes, every output judged by eye and every time in the gate
register; that gate is this release's benchmark, per model. In short:

  • The Triton engine imports nbxtensor, not PyTorch: --triton runs on nbxtensor, Triton and NumPy, and PyTorch is
    no longer a required dependency; the engine, the library and five reference models run in an environment where torch
    is not installed, with no attempt to import it.
  • Every kernel setting is certified per launch key and per card class: the keys a model forms are derived from its
    graph without running it, certified once against a float64 oracle, and served certified-only — a missing key is an
    error naming its census row, never a sweep in the middle of a run.
  • The V100's tensor cores (m8n8k4) carry the matrix products where certified faster: GEMMs, the mixture-of-experts
    prefill projections, fp32 1 x 1 convolutions and transposed 1-D convolutions.
  • The replay fixes: a replayed component returns its views in order, a long video's encoding no longer runs the
    card out of memory at its replay, a replay plan recorded against an old KV cache is retired, a layer-streamed piece
    that frees its inputs is not replayed, and the replay's check says what it observed — a match with the normal path on
    one step, not a proof of the output.
  • Apple Metal on the Triton engine: nbxtensor's Metal driver runs the engine on Apple GPUs, with its own certified
    directory and a vendor file of measured facts (the conv3d workspace, the arena factor, the reload rate); Triton for
    Metal is installed by python -m nbxtensor install-metal (nbxtensor 0.0.4).
  • The TilingEngine cuts only where the graph proves the cut exact, refuses what it cannot place instead of falling
    back, and no longer reads host switches.
  • The light install runs as installed: pip install neurobrix is the Triton engine alone (1.48 GiB installed),
    pip install "neurobrix[pytorch]" adds the PyTorch engine; a run with no mode flag takes the engine that is installed
    and says which on its launch line.
  • Prism plans from the machine's facts: the hardware profile and vendor files carry what it prices (measured
    reserves, reload rates, workspaces), a busy machine is re-planned on the memory really free with a clear warning, and
    a model is refused from one place only when no plan fits.
  • Prism prices a PyTorch-engine load as the loader performs it: one tensor at a time per worker, the pinned host
    cache emptied after each component, a host-held component loaded unpinned then copied into one registered block —
    where it priced every component read and pinned whole (Qwen3-Coder-30B: 115 GB of loading priced, 62.5 GB measured).
  • Certify your GPU and share it: the README's four steps and the certificate pull request template.
  • AMD and Intel GPUs in the Triton engine (nbxtensor's HIP and Level Zero drivers, written, not yet run on a card):
    vendor files for AMD CDNA4, RDNA3 and RDNA4 and Intel Data Center GPU Max, Arc A and B and Lunar Lake, and
    neurobrix doctor names, for every GPU on the PCI bus, the runtime library and Triton backend it lacks and the
    vendor's commands that install them.

Install: pip install neurobrix — the Triton engine, no PyTorch (1.48 GiB installed on a clean python:3.10-slim).
With the PyTorch engine as well: the PyTorch build for your hardware, then pip install "neurobrix[pytorch]" (6.82 GiB
with torch 2.14.1+cu126). On an Apple-silicon Mac (macOS 15 or later, Python 3.12): pip install neurobrix, then
python -m nbxtensor install-metal, which installs Triton for Metal from
NeuroBrix/triton-ext (the AppleGPU backend by imperatormk,
triton-lang/triton-ext#126). neurobrix doctor says what each engine needs on your machine.

The full list of changes (75 added, 97 changed, 2 removed, 330 fixed) is in the
changelog.