Skip to content

nbxtensor 0.0.2

Choose a tag to compare

@benkelaya benkelaya released this 06 Nov 17:03
· 171 commits to main since this release

Added

  • backend.launcher.prewarm_compiler(): computes Triton's cache key (triton_key, the sha256 of libtriton and of Triton's sources, about 1.3 s on a V100 host) in a daemon thread, so a process's first kernel compile no longer waits for it; an application calls it at start, while it loads weights.
  • Intel GPUs through Level Zero (backend.level_zero, backend.level_zero_driver; the xpu backend, chosen when the system's Level Zero loader reports a GPU): a runtime over Level Zero that serves the allocator (device and page-locked host memory, in-order command lists as streams, timed events), and a launch driver for the SPIR-V kernels Intel's Triton build compiles, with Intel's own argument ABI; the compile target is built without torch or SYCL (Level Zero's counts, the OpenCL ICD's extensions, the device file's architecture name). Proven against stand-ins of libze_loader and the OpenCL ICD: no Intel card has run it yet.
  • What to install when a GPU is present and unusable (backend.install_help, python -m nbxtensor doctor): for every NVIDIA, AMD or Intel GPU on the PCI bus, whether its runtime library and its Triton backend are installed, with the vendor's install commands when one is not; the refusal when no GPU runtime answers carries the same steps instead of a bare error. The README's "Install on AMD / Intel GPUs" section gives them.
  • Device files for AMD CDNA4 (gfx950), RDNA3 (gfx11xx) and RDNA4 (gfx120x), and Intel Data Center GPU Max (pvc), Arc A-series and Flex (dg2), Arc B-series (bmg) and Lunar Lake (lnl), every value with its source.
  • A HIP launch driver for AMD GPUs (backend.launcher.HipDriver, chosen when the runtime detects HIP): the HIP module API through the HIP runtime library the runtime already loaded, the launch Triton's own AMD launcher performs (hsaco code objects, hipModuleLaunchKernel, the kernel's arguments then the scratch pointers, a block of the code object's wavefront times the kernel's warps), one module per device and code object, the argument count read from the code object's own metadata and a list of another length refused, shared memory past the device's per-block maximum refused as Triton's own AMD launcher refuses it, the wavefront read from the code object and the device file (never defaulted), the launcher contract's compile surface shared with the CUDA driver, every missing symbol refused by name; page-locked host memory is asked of hipHostMalloc with its flags word. Its tests compile real Triton kernels for gfx90a, gfx942, gfx950, gfx1100 and gfx1201 on a machine without an AMD card and launch them through a stand-in HIP library: no AMD card has run it yet.

Removed

  • Nine environment switches that changed a kernel: NBX_MM_MAX_OUTPUT_ELEMS (the matrix product's output split, fixed by an int32 element offset), NBX_FLASH_BLOCK_M_VOLTA (Volta's flash Q tile, 64, the measured floor of correctness), NBX_CONV3D_EAGER_FREE_BYTES (when a 3-D convolution frees its buffers early), NBX_FLASH_NUM_WARPS and NBX_FLASH_NUM_STAGES (the flash kernel's launch, which its device file's flash rows state), NBX_DGEMV_ADAPT (the dequantising GEMV's width adaptation, set by dequant_gemv.min_programs), NBX_D128_DETOUR (the power-of-two head-dim detour, decided by flash.pow2_head_dim_correct; launch_keys.flash_headdim_detour loses its enabled argument), NBX_DEPTHWISE_DISABLE (the depthwise convolution route) and NBX_GC_ON_OOM (a garbage-collecting retry of a refused allocation, which is raised, never rescued). A value a kernel needs is a fact of its device file or of a certificate, never a host setting.

Changed

  • A certified file is parsed and validated once, then read from its compiled index (ops.certified_index): the directory stays JSON, the source contributors edit; the first launch after a change validates a file and writes, per memory class, a binary index of the entries that class is served (length-prefixed keys and certifications, a SHA-256 trailer, never a pickle), named by the fingerprint of everything the validation read; later launches read the index and parse a certification only when its key is asked. A corrupt index is refused by name.
  • The launch recorder receives each pointer's extent (backend.launcher.set_launch_recorder: fn(prepared, grid, params, extents)): the byte extent of each pointer's tensor that a driver binding by extent is handed, None for a driver that binds by address — a replay can now launch exactly what was launched.
  • The README opens with what the library is and pip install nbxtensor: the 0.0.1 README began its installation with pip install -e nbxtensor from the engine's repository, which PyPI showed as the package's description; it now opens with its title, what it does, the install from PyPI (the repository install is below, for contributors), a GPU example its tests run, why the library exists (no PyTorch, any Triton GPU, the V100's tensor cores back in use with their measured gains, planned memory, certified tuning) and the hardware tested and expected, and it links its API and backend references in the NeuroBrix repository, where they live (they were relative links to files the library's repository does not hold). The package's one-line description no longer names a CPU backend, which the library does not have.

Fixed

  • CDNA3's shared memory per block is 64 KB, not 160 KB (the device file had CDNA4's figure; Triton 3.8's AMD target features state 64 KB for gfx942): an autotune config needing more is no longer kept for an MI300.