Skip to content

basal 2.8.1

Latest

Choose a tag to compare

@rkinas rkinas released this 15 Nov 17:20

basal 2.8.1 (15 November 2026). One fix, for the -FP8 checkpoints. If you serve an -FP8 model with basal 2.8.0, please
update.
Every other start answers as in 2.8.0: the answers were checked byte for byte against 2.8.0 on an H100 and on
an RTX PRO 6000 Blackwell (basal-omni-1.0-mini in --mode fast, --mode eager and with --media on;
basal-omni-1.0-mini-FP8, basal-omni-pico and basal-2.0-mini-FP8 in --mode fast; basal-2.0-4.5B).

What was wrong. basal 2.8.0 loaded an NVIDIA ModelOpt FP8 checkpoint in --mode eager and --mode mps without its
scales and served wrong answers, with no error: the log showed only a warning from transformers ("Unknown quantization
type, got modelopt"). On basal-omni-1.0-mini-FP8 in --mode eager, 24 of 68 answers on a test set kept the bf16 model's
top answer.

Who was affected.

  • The checkpoints: every public -FP8 repository: basal-omni-1.0-mini-FP8, -ultra-FP8 and -max-FP8;
    basal-2.0-mini-FP8, -4.5B-FP8 and -max-FP8; the basal-1.5 and basal-1.0 -FP8 checkpoints.
  • Without --mode: basal-omni's -FP8 checkpoints ran in eager on every NVIDIA GPU except Hopper (H100, H200)
    with the flash_attn_3 package and the Blackwell workstation GPUs (RTX PRO 6000, RTX 50xx), for example A100, A10,
    L4, L40S, RTX 6000 Ada, RTX 30xx and 40xx, H100 without flash_attn_3, B200 and Turing GPUs. On a machine without a
    GPU every -FP8 checkpoint ran in eager, and on Apple Silicon without mlx in mps.
  • If you served an -FP8 checkpoint in --mode eager or --mode mps (or without --mode on one of those
    machines), please run those requests again with 2.8.1, or with the bf16 model.
  • Not affected: --mode fast, --mode fp8 and --mode vllm read the FP8 layers and served the right answers, as
    they do now.

What 2.8.1 does. Such a start now stops at once, before any weights download, with a message naming --mode fast
(it reads the FP8 layers on a CUDA GPU), --mode vllm, or the bf16 model. Without --mode, a machine whose default
would be one of those modes stops the same way. The -NVFP4 checkpoints, which failed to load outside --mode vllm,
get the same message. Details: WHATS_NEW.

Install:

uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.1.tar.gz"

The prepared/ start-up kernels shipped in the model repositories for 2.8.0 load in 2.8.1 unchanged.