Repository navigation
basal 2.8.1 (15 November 2026). One fix, for the -FP8 checkpoints. If you serve an -FP8 model with basal 2.8.0, please
update. Every other start answers as in 2.8.0: the answers were checked byte for byte against 2.8.0 on an H100 and on
an RTX PRO 6000 Blackwell (basal-omni-1.0-mini in --mode fast, --mode eager and with --media on;
basal-omni-1.0-mini-FP8, basal-omni-pico and basal-2.0-mini-FP8 in --mode fast; basal-2.0-4.5B).
What was wrong. basal 2.8.0 loaded an NVIDIA ModelOpt FP8 checkpoint in --mode eager and --mode mps without its
scales and served wrong answers, with no error: the log showed only a warning from transformers ("Unknown quantization
type, got modelopt"). On basal-omni-1.0-mini-FP8 in --mode eager, 24 of 68 answers on a test set kept the bf16 model's
top answer.
Who was affected.
- The checkpoints: every public
-FP8repository: basal-omni-1.0-mini-FP8, -ultra-FP8 and -max-FP8;
basal-2.0-mini-FP8, -4.5B-FP8 and -max-FP8; the basal-1.5 and basal-1.0-FP8checkpoints. - Without
--mode: basal-omni's-FP8checkpoints ran ineageron every NVIDIA GPU except Hopper (H100, H200)
with theflash_attn_3package and the Blackwell workstation GPUs (RTX PRO 6000, RTX 50xx), for example A100, A10,
L4, L40S, RTX 6000 Ada, RTX 30xx and 40xx, H100 withoutflash_attn_3, B200 and Turing GPUs. On a machine without a
GPU every-FP8checkpoint ran ineager, and on Apple Silicon without mlx inmps. - If you served an
-FP8checkpoint in--mode eageror--mode mps(or without--modeon one of those
machines), please run those requests again with 2.8.1, or with the bf16 model. - Not affected:
--mode fast,--mode fp8and--mode vllmread the FP8 layers and served the right answers, as
they do now.
What 2.8.1 does. Such a start now stops at once, before any weights download, with a message naming --mode fast
(it reads the FP8 layers on a CUDA GPU), --mode vllm, or the bf16 model. Without --mode, a machine whose default
would be one of those modes stops the same way. The -NVFP4 checkpoints, which failed to load outside --mode vllm,
get the same message. Details: WHATS_NEW.
Install:
uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.1.tar.gz"The prepared/ start-up kernels shipped in the model repositories for 2.8.0 load in 2.8.1 unchanged.