v0.1.0 — all three models above llama.cpp at short context

First tagged release. bw24 is a from-scratch Rust+CUDA LLM inference engine built for one machine — RTX 5090 Laptop (sm_120a, 24 GB) — with llama.cpp on the same rig as the floor to beat. Every optimization is gated on bit-exactness (kernel-check vs CPU reference, prefill/decode argmax match, K=1..8 speculative self-consistency).

State at v0.1.0 (all numbers same-day, both engines, full power)

Plain decode, tg128 (bw24 / llama.cpp):

• d512: 9B 132.7/124.6 (1.07x), 27B 47.7/43.5 (1.10x), 35B MoE 173.4/170.5 (1.02x) — all three models above llama.cpp
• d6257: 9B 124.5/119.6 (1.04x), 27B 44.9/42.0 (1.07x), 35B 158.5/159.9 (0.99x)

Speculative (MTP head, raw-prompt protocol, p1/p2/p3):

• 9B 243/195/162 vs 186/158/155 · 27B 108/91/79.5 vs 86.4/89.9/73.2 · 35B 203/201/185 vs 215/208/202

Highlights in this release:

• FA_V2 tile-batched online-softmax attention decode (default) — fixed the engine-wide depth slope; the 9B deep-context cell flipped from 0.91x to 1.04x
• Fast path is the default — no environment flags needed; flags remain only for runtime parameters, machine config, and rollback seams
• FR-Spec vocabulary trims + frspec_rank builder (per-tokenizer artifact, published on HF), zero-draft rounds, per-content-class draft depth
• FP8-activation prefill GEMM for F8-native checkpoints (BW24_PP_FP8, VRAM-budgeted)
• Safetensors loading (NVIDIA modelopt NVFP4, MiniMax-M3 REAP50 121GB via NVMe expert streaming)
• Reproduction artifacts: https://huggingface.co/Avifenesh/bw24-bench

Known gaps (tracked in research/tune-data/): prefill 0.55-0.74x of llama.cpp (precision-ceiling decomposition done; exactness-safe levers named), 35B speculative 0.92-0.97x (verify m-scaling curve measured), 35B deep-context 0.99x.