NeuroBrix 0.6: the engine now runs without PyTorch

[benkelaya]

NeuroBrix 0.6 is out, and the current release is 0.6.2. It is the first stable release of the engine.

pip install neurobrix              # the light install: Triton engine, no PyTorch
python -m nbxtensor install-metal  # once, on a Mac with Apple Silicon

What's new in 0.6

• A Triton engine with no PyTorch. It is built on nbxtensor, our own tensor library. The light install is 1.48 GB, against 6.82 GB with PyTorch. If you also want the PyTorch engine: pip install "neurobrix[pytorch]".
• 49 models in 9 families: LLMs and code, vision-language and omni, speech recognition, TTS, image, video and upscaling. Every model is validated on every release, on both engines.
• Faster than PyTorch on 18 of the 49 models on a V100, up to 5x on Janus-Pro-7B. The V100 tensor cores are back, through hand-written m8n8k4 kernels.
• Linux, Windows and macOS. NVIDIA, AMD, Intel and Apple GPUs.
• Triton on Apple Silicon through NeuroBrix's own build (NeuroBrix/triton-ext), with 18,552 kernel settings certified for Apple GPUs.
• Certified kernel tuning. Kernel settings are certified once per GPU class, not autotuned on your machine. You can certify your own GPU and share the result: see Certify your GPU and share it.

Release notes: v0.6.2 · CHANGELOG · Models: neurobrix.es/models

Tell us what you run it on. Questions go to Q&A, ideas to Ideas, and anything you built to Show and tell.

— Hocine Benkelaya, NeuroBrix