v0.9.5

What's Changed

• fix(core): only honor tool-call and reasoning delimiters emitted as special tokens by @EricLBuehler in #2455
• feat(models): add support for Qwen3.8-Flash-Next by @EricLBuehler in #2462
• feat(cuda): support CUDA 13.4 by @EricLBuehler in #2481
• fix(cuda): support GCC 13+ in radix top-k kernel by @wsj040303 in #2461
• fix(quant): avoid deadlock while batching ISQ output jobs by @mikemikimike in #2463
• feat(metal): make Metal Paged Attention the default backend by @EricLBuehler in #2493
• chore: v0.9.4 -> v0.9.5 by @EricLBuehler in #2498

New Contributors

• @wsj040303 made their first contribution in #2461
• @mikemikimike made their first contribution in #2463

Full Changelog: v0.9.4...v0.9.5

v0.9.4

What's Changed

• chore(deps): fix open Dependabot security alerts by @EricLBuehler in #2444
• fix(server): accept only a bare file name for calibration save_cimatrix by @EricLBuehler in #2445
• fix(server): judge IPv6 transition addresses by their embedded IPv4 in the media guard by @EricLBuehler in #2446
• fix(ui): validate chat ids before building chat file paths by @EricLBuehler in #2447
• fix(server): bound decoded GIF frames and audio samples by @EricLBuehler in #2448
• fix(search): share the remote fetch guard with the web search and extraction tools by @EricLBuehler in #2449
• chore(deps): bump candle to e65eb1de and cutile to 0.3.1 by @EricLBuehler in #2450
• fix(quant): restore CUDA builds and inference after the candle bump by @EricLBuehler in #2451
• chore(release): 0.9.4 by @EricLBuehler in #2452
• fix(core): read the CUDA graph instantiate flag through the cudarc 0.19.10 newtype by @EricLBuehler in #2454

Full Changelog: v0.9.3...v0.9.4

v0.9.3

What's Changed

• feat(core): concurrency fixes and scheduler improvements by @EricLBuehler in #2389
• perf: first-class FP8 and concurrent DFlash decode by @EricLBuehler in #2390
• feat(metrics): add mistralrs_sequences_capacity gauge by @EricLBuehler in #2391
• perf(core): optimize exact stochastic DFlash serving by @EricLBuehler in #2392
• feat(server): align server error codes to openai standard by @EricLBuehler in #2393
• feat(logging): expose separate prefill and decode t/s logging by @EricLBuehler in #2395
• fix(multimodal): keep Qwen media work out of decode by @EricLBuehler in #2396
• fix(ci): green the build for rust 1.98 and metal by @EricLBuehler in #2397
• feat(core): report cached prompt tokens in usage by @EricLBuehler in #2398
• perf(core): improve hybrid FP8 serving by @EricLBuehler in #2399
• feat(server): improve Anthropic Messages and Claude Code support by @matt-minev in #2400
• docs: refresh generated reference artifacts by @matt-minev in #2401
• fix(website): fix handling for clipboard by @EricLBuehler in #2403
• perf(cuda): scale FP8 and DFlash2 serving through C128 by @EricLBuehler in #2404
• feat(website): add details about v0.9.3 release by @EricLBuehler in #2405
• chore(cuda): clippy by @EricLBuehler in #2406
• feat(release): improve reliability, add caching by @EricLBuehler in #2408
• fix(core): align tool chat template JSON rendering by @atirna in #2371
• fix(lfm2): support legacy full_attn_idxs config field by @tugot17 in #2305
• fix(webui): send chat images as data URLs (fix HTTP 500 on image send) by @subin9 in #2338
• feat(core): bump cutile to v0.3.0 by @EricLBuehler in #2410
• feat(core): fp8 tensor-core gemv/gemm, cutile gdn prefill by @EricLBuehler in #2412
• feat(cutile): add fp8 fused moe and a load-time kernel autotuner by @EricLBuehler in #2413
• fix(cuda): restore GB10 builds with cudaforge 0.1.6 by @sizzlecar in #2415
• feat(fp8): support ModelOpt and compressed-tensors checkpoints with cutile kernels by @EricLBuehler in #2416
• feat(quant): add NVFP4 checkpoint loading and cuTile Blackwell acceleration by @EricLBuehler in #2418

New Contributors

• @atirna made their first contribution in #2371
• @tugot17 made their first contribution in #2305
• @subin9 made their first contribution in #2338
• @sizzlecar made their first contribution in #2415

Full Changelog: v0.9.2...v0.9.3

v0.9.2

What's Changed

• fix(fav3): misc fixes relating to FAv3 by @EricLBuehler in #2369
• feat(qwen3.5): implement better video processor by @EricLBuehler in #2368
• feat(server): improve openai server compatability with /v1/responses by @EricLBuehler in #2370
• feat(qwen3.5): add mtp support by @EricLBuehler in #2372
• fix(qwen3.5): better support for hybrid models and auto device map by @EricLBuehler in #2373
• feat(metrics): add engine metrics for scheduler status by @EricLBuehler in #2374
• fix(qwen3.8): fixes around gdn rollback for qwen3.8 by @EricLBuehler in #2377
• feat(mtp): support cuda graphs by @EricLBuehler in #2379
• feat(gdn): in-place pooled recurrent state kernels, fused MLP, f32 gate params by @EricLBuehler in #2382
• feat(cuda): CUDA graph batch buckets by @EricLBuehler in #2384
• feat(metrics): improve engine metrics for ttft/itl and speculative by @EricLBuehler in #2383
• perf(mtp): quantized drafter, device verify, stash-based GDN rollback, size-aware n_predict by @EricLBuehler in #2385
• feat(speculative): DFlash and DFlash 2 block-diffusion draft models by @EricLBuehler in #2386
• perf(dflash): batched drafter forward and adaptive draft depth by @EricLBuehler in #2388

Full Changelog: v0.9.1...v0.9.2

v0.9.1

What's Changed

• feat(install): show suggested commands for installer script by @EricLBuehler in #2315
• feat(uqff): no dummy run for uqff generation pass by @EricLBuehler in #2316
• fix(sampler): return natural-log probabilities from logprobs, not log10 by @dx2102 in #2327
• fix(core): address recent correctness regressions by @EricLBuehler in #2345
• chore(deps): bump serde_with from 3.16.1 to 3.21.0 by @dependabot[bot] in #2349
• feat(lora): add true lora kernels and dynamic adapter support by @EricLBuehler in #2351
• feat(core): support lmhead/embedding quantization by @EricLBuehler in #2280
• fix(quant): misc fixes, warmup metal kernels by @EricLBuehler in #2353
• fix(gemma4): improve audios ordering around text by @EricLBuehler in #2355
• feat(core): major improvements to scheduler & concurrent serving performance by @EricLBuehler in #2354
• feat(core): first-class support for loading GGUFs for all supported models by @EricLBuehler in #2365
• feat(models): support Muse Glimmer by @EricLBuehler in #2366

New Contributors

• @dx2102 made their first contribution in #2327

Full Changelog: v0.9.0...v0.9.1

v0.9.0

What's Changed

• Fix CUDA version extraction in install.sh by @zaqxsw-dev in #2287
• feat(metal): improve runtime and aot compilation system by @EricLBuehler in #2288
• feat(release): split cuda workers for docker release by @EricLBuehler in #2289
• feat(install): cleanup install system for updating by @EricLBuehler in #2290
• feat(distributed): more sophisticated TP fallback and resolution system by @EricLBuehler in #2291
• chore(mistralrs-core): cap tojson indent and remove chat template panic paths by @eharris128 in #2286
• feat(quant): improve uqff resolution per model ids by @EricLBuehler in #2293
• Fix agentic loop not starting for registered tool callbacks by @aeiklorvy in #2279
• feat(models): Support LFM 2.5 (dense + MoE) and LFM 2.5 VL! by @EricLBuehler in #2295
• feat(cuda): preload cuda ptx modules by @EricLBuehler in #2296
• refactor(core): move code exec shims from lib.rs to proper location by @EricLBuehler in #2297
• docs(models): improve supported models readability by @EricLBuehler in #2298
• feat(multi-model): support planning for loading multi-model by @EricLBuehler in #2299
• feat(cuda): update cutlass git revision by @EricLBuehler in #2300
• feat(cpu): aarch64 cpu repacking kernels and performance improvements by @EricLBuehler in #2304
• feat(cpu): optimize decode and prefill performance on x86 and aarch64 by @EricLBuehler in #2311
• fix(tune): detect CUDA backend on Windows by @matt-minev in #2312
• fix(server): keep Responses output item IDs stable by @matt-minev in #2309

New Contributors

• @zaqxsw-dev made their first contribution in #2287
• @eharris128 made their first contribution in #2286
• @aeiklorvy made their first contribution in #2279
• @matt-minev made their first contribution in #2312

Full Changelog: v0.8.23...v0.9.0

v0.8.23

What's Changed

• feat(cuda): support CUDA 13.3 by @EricLBuehler in #2275
• chore: bump candle dep by @EricLBuehler in #2276
• feat(cli): improve --quant and --isq docs in cli by @EricLBuehler in #2277
• feat(gemma4): do not load projections for shared kv layers by @EricLBuehler in #2281
• feat(quant): add isq executor and planning by @EricLBuehler in #2283
• feat: add Hunyuan v1 dense and MoE support by @ASheng1019 in #2268
• feat(install): fix handling when there are preexisting installs by @EricLBuehler in #2284
• feat(gdn): add isq support by @EricLBuehler in #2285
• Fix reversed FCFS priority in PagedAttentionScheduler preemption by @pjdurden in #2250
• Validate GGUF special token ids against vocab to prevent OOB panic by @pjdurden in #2282

New Contributors

• @ASheng1019 made their first contribution in #2268
• @pjdurden made their first contribution in #2250

Full Changelog: v0.8.22...v0.8.23

v0.8.22

What's Changed

• feat(cuda): improve release matrix, better errors, explicit cutile handling by @EricLBuehler in #2274

Full Changelog: v0.8.21...v0.8.22

v0.8.21

What's Changed

• fix(core): remove local repo list cache by @EricLBuehler in #2270
• feat(core): improve errors and explanations during model resolution by @EricLBuehler in #2271

Full Changelog: v0.8.20...v0.8.21

v0.8.20

What's Changed

• feat(ci): improve release workflow binary upload reliability by @EricLBuehler in #2267
• feat(quant): improve uqff generation device mapping handling by @EricLBuehler in #2269

Full Changelog: v0.8.19...v0.8.20