v0.9.1

What's Changed

• feat: implement Motif model by @AlpinDale in #1439
• [V1] feat: support fp8 kv on ampere through flashinfer by @AlpinDale in #1440
• fix: enable Kobold API by default by @AlpinDale in #1441
• feat: add DeepConf sampling by @AlpinDale in #1442
• build: add option to disable flash attn compilation by @AlpinDale in #1443
• [PP] feat: optimize PP performance through throttling by @AlpinDale in #1447
• chore: sync to upstream 4071c76 by @AlpinDale in #1448
• feat: support encoder data parallel for MiniCPM-V by @AlpinDale in #1449
• feat: enable LoRA support for DeepSeek models by @AlpinDale in #1450
• fix: add reorder_batch to AttentionMetadataBuilder by @AlpinDale in #1451
• chore: move out freezing_value/cuda_event init outside try/finally block by @AlpinDale in #1452
• chore: better type hint for rearrange return value in eplbstate by @AlpinDale in #1453
• chore: faster LoRA startup by @AlpinDale in #1454
• fix: truncate_prompt_tokens type hint by @AlpinDale in #1455
• feat: allow passing multi_modal_uuids as multimodal identifiers by @AlpinDale in #1456
• fix: vocab_size check by @AlpinDale in #1457
• feat: support KV events from connectors by @AlpinDale in #1458
• fix: AutoGPTQ Qwen3-MoE models by @AlpinDale in #1459
• fix: avoid redundant copy for enc-only models by @AlpinDale in #1460
• fix: loading GPTQ 3-bit models by @AlpinDale in #1461
• chore: move fast prefill logic to a separate method by @AlpinDale in #1462
• fix: update constructors and type hints for multimodal input handling by @AlpinDale in #1463
• chore: update import for torch inductor configuration in CPU model runner by @AlpinDale in #1464
• chore: migrate Phi4 inputs to TensorSchema by @AlpinDale in #1465
• feat: IO processor plugin for pooling models by @AlpinDale in #1466
• fix: add support for <tool_call> format in streaming mode for xlam by @AlpinDale in #1467
• fix: allow FP16 inference on Turing and below by @AlpinDale in #1468
• build: upgrade DeepGEMM by @AlpinDale in #1469
• feat: Gemma3n audio endpoint support by @AlpinDale in #1470
• feat: support KeyeVL-1.5 model by @AlpinDale in #1471
• chore: minor code simplification for spec decode by @AlpinDale in #1472
• fix: IO processor plugin fixes by @AlpinDale in #1473
• feat: support DP for Kimi-VL ViT by @AlpinDale in #1474
• chore: move logprob to a separate module by @AlpinDale in #1475
• [kernel] feat: support FP32 for mamba by @AlpinDale in #1476
• fix: blip2 inference by @AlpinDale in #1477
• chore: remove runtime checks for pooling params by @AlpinDale in #1478
• chore: migrate Ovis to TensorSchema by @AlpinDale in #1479
• feat: add online FP8 support for XPU by @AlpinDale in #1480
• chore: migrate interns to TensorSchema by @AlpinDale in #1481
• feat: support DP for GLM-4.5V ViT by @AlpinDale in #1482
• fix: gemma3n batched audio by @AlpinDale in #1483
• fix: EXAONE4 model RoPE by @AlpinDale in #1484
• chore: add logit_bias/sigmoid_normalize support for classification models by @AlpinDale in #1485
• fix: transform_config parsing in compressed tensors by @AlpinDale in #1486
• fix: pack_factor -> packed_factor in linear layers by @AlpinDale in #1487
• chore: deprecate TPOT in favor of ITL by @AlpinDale in #1488
• fix: weight loading for Apertus model by @AlpinDale in #1489
• fix: only print profiler results on rank 0 by @AlpinDale in #1490
• fix: cast offsets tensor bn to tl.int64 to avoid GPU segfault by @AlpinDale in #1491
• fix: DeepSeek-R1 accuracy by setting routed_scaling_factor=1.0 by @AlpinDale in #1492
• fix: LoRA logits on XPU by @AlpinDale in #1493
• feat: multi-threaded model weights loader by @AlpinDale in #1494
• feat: fully async model execution by @AlpinDale in #1495
• feat: enable pytorch symmetric memory for A100 GPUs by @AlpinDale in #1496
• [kernel] feat: vectorized kernels by @AlpinDale in #1497
• chore: upgrade xgrammar to 0.1.23 by @AlpinDale in #1499
• feat: enable request-level logits processor in the batch-level logit-procing by @AlpinDale in #1500
• fix: compile warning for w4a8_mm_entry.cu by @AlpinDale in #1501
• fix: add check for dual chunk attn by @AlpinDale in #1502
• fix: double mul for dots1 and GLM4 MoE by @AlpinDale in #1503
• chore: remove NCCL cumem env var override by @AlpinDale in #1504
• feat: enable heterogenous TP for Nixl KV connector w/ flashinfer by @AlpinDale in #1505
• chore: support add_generation_prompt in embeddings endpoint by @AlpinDale in #1506
• refactor: simplify weight handling in MiniMaxText01RMSNormTP class by removing dead code by @AlpinDale in #1507
• fix: division by zero in triton_attn by @AlpinDale in #1508
• feat: enable full CUDA graph support for PLaMo2 on V1 by @AlpinDale in #1509
• feat: add support for Qwen3-Next model and add Flash Linear Kernels by @AlpinDale in #1510
• Fix for logit bias crash (probably) by @50h100a in #1511

Full Changelog: v0.9.0...v0.9.1