v0.23.0

v0.23.0 - 2026.08.16

We're excited to announce the official vLLM Ascend v0.23.0 release, aligned with upstream vLLM v0.23.0. This note summarizes the cumulative user-facing changes since the previous official release, v0.18.0, including the v0.19.1rc1, v0.20.2rc1, v0.21.0rc1, v0.22.1rc1, and v0.23.0rc1 development cycles. Please follow the official documentation to get started.

PR references marked with † were merged into the v0.23.0 release branch after v0.23.0rc1.

Highlights

• Ascend 950 and DeepSeek V4: Added end-to-end DeepSeek V4 support on Ascend 950, including DSA attention, MTP, piecewise graph execution, distributed inference, sparse attention, CPU binding, and MXFP quantization and communication paths. #9757 #9935 #10236 #11014
• Model and hardware coverage: Expanded support and deployment guidance for GLM-5.2, GLM-4.7-Flash, Qwen3.5/Qwen3.6, Qwen3-ASR, Qwen3-Omni, Bailing MoE, Gemma4, Step3, and MiniMax 2.x models across A2, A3, Ascend 950, and Atlas 300I DUO. GLM-5.2 supports long-sequence inference up to 1M tokens on Atlas 800 A3. #8657 #9560 #10441 #10697 #11091 #11264 #12115
• Context parallelism and sparse attention: Added SFA DCP with a replicated indexer, compact KV gather, C8 support, and device-side metadata paths for long-context and P/D-disaggregated deployments. #9638 #9809 #11819 #11871 #11981
• KV-cache lifecycle and offload: Added hybrid/Mamba attention prefix caching; CPU and SSD offload in AscendStore that covers all backends. #8743 #9533 #9731 #10393
• Graph and speculative execution: Added FULL_AND_PIECEWISE graph mode, which is enabled by default and requires no manual configuration; DFlash FULL_DECODE_ONLY; zero-bubble async scheduling; P-Eagle and PARD; and expanded MTP/Eagle3 support. #7640 #8118 #9572 #10042 #10566

Features

• Added multimodal DFlash, FlashComm support for Qwen VL/MoE models, and PCP-aware multimodal reasoning. #7486 #7897 #8038 #9340
• Added HCCL weight transfer for reinforcement-learning workloads and D2D NetLoader support for speculative draft models. #9152 #9893
• Expanded Model Runner V2 with initial MoE and Eagle support. #7885 #7922
• Expanded EPLB with additional observability and dynamic load-balancer examples. #9536 #10627
• Extended C8 INT8 KV cache to sparse-attention paths with packed layouts and a DCP replicated indexer, and added W8A8FP8 and W4A16 MXFP quantization paths for Ascend 950. #10236 #11014 #11846 #11871

Hardware and Operator Support

• Added and optimized recurrent GDN, causal Conv1D, sparse-attention, LightningIndexer, compressor, and fused quantization operators. #7798 #7926 #9382 #9491 #9825 #10730
• Expanded Atlas 300I DUO support for Qwen3.5, Qwen3.6, Qwen3-ASR, Qwen3-VL, quantized MoE paths, MTP, and graph execution. #7674 #7725 #10309 #12115 #13262†
• Added Python 3.12 support and moved release images to Python 3.12. #9558

Performance

Unless stated otherwise, these optimizations are selected automatically for the targeted path and need no additional configuration.

• Replaced npu_fusion_attention with _npu_flash_attention_unpad for supported A2/A3 attention workloads. It is selected automatically; no manual setting is needed. #8671
• Avoided projecting unused tail KV tokens during MLA prefill with PCP. Enable PCP with --prefill-context-parallel-size; the optimization then applies automatically. #8787
• Reduced scheduler issuance bubbles for workloads using asynchronous scheduling. Enable it with --async-scheduling. #8766
• Added zero-bubble scheduling for asynchronous speculative decoding. Enable it with --async-scheduling together with a speculative decoding configuration. #7640
• Batched KV-cache offload copies with aclrtMemcpyBatchAsync for CPU-offload workloads. Configure KV cache CPU offload as documented; batching is automatic within that path. #7819
• Reduced PCP/DCP KV-cache all-gather traffic by selecting the required blocks before communication. Enable PCP or DCP with --prefill-context-parallel-size or --decode-context-parallel-size; no separate optimization switch is needed. #8050
• Optimized split_qkv_tp_rmsnorm_rope kernels for supported quantized model paths. Kernel selection is automatic; no manual setting is needed. #8059 #9830
• Removed prefill host-device synchronization in Qwen3-Next and Qwen3.5 paths. It applies automatically to those models. #7967
• Reduced SFA prefill KV all-gather communication for PCP/DCP. Enable the corresponding context-parallel mode; the optimized communication path is automatic. #8043
• Added a Triton penalty kernel for requests using repetition, frequency, or presence penalties. It is selected automatically when penalties are requested. #7569
• Optimized Model Runner V2 temperature and top-k log-softmax kernels. They are selected automatically for sampling workloads on Model Runner V2. #8083
• Optimized the Model Runner V2 min-p kernel. It applies automatically when min-p sampling is requested. #8243 #7767
• Added a Model Runner V2 Triton kernel for bad-word filtering. It applies automatically when bad_words is supplied. #8030
• Optimized the Model Runner V2 bincount kernel. It is selected automatically for sampling paths that require token counts. #7757
• Optimized the Model Runner V2 ranks kernel. It is selected automatically for the corresponding sampling path. #7767
• Reduced avoidable Triton recompilation caused by runtime function parameters. The cache-friendly path is automatic; no manual setting is needed. #7481 #7483
• Reused equivalent HCCL process groups to reduce distributed initialization overhead. Reuse is automatic for matching groups. #7654
• Deferred CPU binding until worker warmup completes to avoid interfering with initialization. CPU binding is enabled by default on supported ARM servers; no manual setting is needed unless it was explicitly disabled. #7829
• Converted eligible Conv3D operations to linear operations when kernel size equals stride. Conversion is automatic for matching models. #8318
• Optimized MoE routing on Ascend 310P. It applies automatically to supported 310P MoE workloads. #9105
• Added NZ-format W4A8 MoE compressed tensors for supported quantized models. Load a compatible W4A8 checkpoint with Ascend quantization; format selection is automatic. #9625
• Optimized irregular-mask construction for PCP/DCP with speculative decoding. Enable context parallelism and speculative decoding; no separate optimization switch is needed. #9678
• Reworked reduce sampling for DFlash and MTP. This remains experimental and disabled by default; enable it with additional_config.enable_reduce_sample=true for distributed greedy, top-k/top-p, or rejection sampling. #8308 #9735
• Added multistream compute-communication overlap for DeepSeek V4 DSA compressor, indexer-select, CV-parallel, and pure-prefill paths. Enable FlashComm1 and the applicable DSA-CP/overlap configuration; the optimized subpaths are then selected automatically. #9433 #9441 #9450 #9504 #9530 #10518
• Reused DeepSeek V4 DSA topk_indices across decode steps through IndexCache. It applies automatically to eligible DSA decode workloads. #9390
• Removed a host-device synchronization point from PIECEWISE graph execution. FULL_AND_PIECEWISE is the default graph mode, so no manual configuration is needed. #9025
• Optimized shared-expert overlap timing in FusedMoE. Enable the supported shared-expert overlap configuration; timing optimization is automatic within that path. #9413
• Skipped unnecessary slot-mapping computation for Mamba groups in hybrid-cache models. It applies automatically when the model uses a Mamba cache group. #10492
• Built single-rank DSA compressor metadata on device to remove CPU construction and synchronization. It applies automatically to eligible non-CP DSA compressed-KV paths. #10741
• Vectorized SFA local-sequence-length computation to remove per-request NPU-to-CPU synchronization. It applies automatically to SFA workloads. #11816
• Replaced the SFA DSA-CP output merge's full all-gather with token-sharded all-to-all to reduce communication and peak receive-buffer use. Enable FlashComm1 and additional_config.enable_dsa_cp=true; no separate optimization switch is needed. #12137
• Reduced restore and output-merge overhead in the PCP FlashAttention path. Enable PCP with --prefill-context-parallel-size; the optimized restore path is automatic. #11842
• Avoided host-to-device synchronization while building context-parallel speculative-proposer metadata. Enable context parallelism and speculative decoding; the optimized metadata path is automatic. #11862
• Snapshotted query start locations before asynchronous host-to-device copies to preserve non-blocking metadata transfers. It applies automatically to asynchronous copy paths. #12071
• Added fused W4A8 MoE dispatch, FFN, and combine to overlap communication with computation. Load a supported W4A8 MoE checkpoint with Ascend quantization; fused-kernel selection is automatic. #7779
• Added asynchronous all-gather for DSA-CP output-projection TP weights. Enable FlashComm1 and additional_config.enable_dsa_cp=true; the asynchronous path is automatic. #10694
• Improved DeepSeek V4 prefix-cache hit rates when MTP is enabled. Prefix caching is enabled by default in vLLM V1; no flag is needed unless it was previously disabled. #11107
• Reused prebuilt chunk host metadata to reduce synchronization overhead for Qwen3.5 and MiniMax-M2.5 workloads. It applies automatically to the affected model paths. #9310
• Parallelized Mooncake KV receive handling for P/D-disaggregated deployments. Configure the Mooncake KV-transfer connector; receive parallelism is automatic. #10548
• Optimized AscendStore key construction and KV Pool miss handling to reduce host overhead. Configure AscendStoreConnector; the optimized paths are selected automatically. #12783†
• Reused grouped block hashes across AscendStore query, save, and load operations to reduce repeated host-side hashing. Configure AscendStoreConnector; reuse is automatic. #13169†

Stability and Bug Fixes

• Fixed GDN state and graph-dispatch accuracy regressions across P/D, PCP, MTP, and DCP, including one-token stateful prefill handling. #11195 #11893 #12027 #12255†
• Fixed Qwen3.5/Qwen3.6 speculative-decoding accuracy, Mamba prefix-cache corruption, block-table overflow, and Atlas 300I DUO graph failures. #11337 #11408 #11353 #11659 #11920 #12038
• Fixed GLM-5.1 IndexCache weight loading and a GLM-4.7-Flash first-request IndexError with MTP and layerwise Memcache. #11363 #11829
• Fixed Qwen MoE routing overflow and shared-expert gate failures, Qwen3-Omni ModelSlim W8A8 checkpoint loading, and Qwen3-VL rotary-embedding copy races on Atlas 300I DUO. #11391 #11730 #12321 #11679 #12132
• Fixed the DeepSeek-R1-0528 W8A8 shared-expert no-clamp accuracy path and malformed streamed tool calls or TP8+EP startup for MiniMax-M2/M2.5. #11775 #11505
• Fixed DeepSeek V4 MXFP routing precision, W4A16MXFP communication accuracy, and eager draft compilation configuration isolation. #11663 #11718 #12587† #12722†
• Fixed KV-transfer ordering and TP-shard consistency, Mooncake grouping, AscendStore hash/lease/lookup handling, and layerwise KV Pool failures. #11887 #12252 #12371† #12797† #12819†
• Fixed DCP/DP service hangs, limited the recompute scheduler to decode nodes, and delayed AscendStore initialization until the first real decode request. #12034 #11490 #11673
• Disabled unsupported shared-expert multistream overlap with fused MC2 and fixed low MTP acceptance for SFA with DSA-CP and multiple speculative tokens. #12245 #10878
• Fixed AscendStore compatibility with pipeline parallelism and surfaced backend failures. #12762†
• Fixed invalid-GVA handling for layerwise KV Pool transfers. #12643†
• Fixed A5 BF16 mm_reduce_scatter communication-mode selection and restricted FP8 quantization detection to A5 devices. #12826† #12895†
• Packed SFA DSA-CP KV tensors into one all-gather to correct and streamline the communication path. #12867†
• Disabled unsupported npugraph_ex compilation automatically on Atlas 300I DUO and fixed low-level rotary and sparse-attention tiling/metadata issues. #12366† #12446† #12841†
• Fixed npu_dequant_swiglu_quant precision for small token shapes and propagated return_lse to QuantSFA operators. #12913† #13197†
• Removed unnecessary Memcache lazy initialization and hardened AscendStore transfers by restoring synchronous saves, reporting asynchronous load failures to the scheduler, and using grouped block sizes in transfer threads. #13028† #13024† #13099† #13110†
• Removed -Werror from custom-operator CMake builds so compiler warnings do not fail release builds. #13095†

Dependencies

• Upstream vLLM: v0.23.0.
• Python: >= 3.10, < 3.13.
• CANN: 9.1.0 for A2, A3, and Ascend 950; refer to the Atlas 300I DUO installation guide for its platform-specific CANN package. #13421†
• PyTorch / torch_npu: 2.10.0 / 2.10.0.post4.
• Triton Ascend: 3.2.2 for A2, A3, and Ascend 950; Triton Ascend is not supported on Atlas 300I DUO.
• Mooncake: 0.3.11.post1 in the release images.

Deprecation and Configuration Changes

• The former enable_sparse_c8 option was split into enable_sparse_sfa_c8 and enable_sparse_li_c8; update --additional-config according to the sparse-attention components in use. #12351
• Migrate FlashComm1 deployments from VLLM_ASCEND_ENABLE_FLASHCOMM1 to additional_config.enable_flashcomm1. #9064
• VLLM_ASCEND_ENABLE_CONTEXT_PARALLEL was removed during the AscendConfig migration. DSA-CP is now controlled by additional_config.enable_dsa_cp; deployments that previously relied on FlashComm1 implicitly enabling DSA-CP must explicitly enable both options. #9668 #9697 #9910
• Sequence Parallelism is marked unavailable for all current model categories in the v0.23.0 support matrix; deployments that used it in v0.18.0 should move to FlashComm1 where applicable. #12860†
• ASCEND_BUFFER_POOL was removed. Use ASCEND_ENABLE_USE_FABRIC_MEM=1 or HCCL_INTRA_ROCE_ENABLE=1 according to the hardware and deployment path. #13834†

Ready to Deprecate

The following features and optimizations are planned for deprecation in a future release:

• Layer sharding.
• FlashComm2.
• The FlashComm3 multistream-overlap gate.
• Hamming sparse.
• Asynchronous exponential overlap.
• Matmul all-reduce and matmul all-reduce RMSNorm fusions.
• Weight prefetch.
• Dynamic-batch SLO.
• KV offload in KV Pool.
• Fused MC2 mode 2 (enable_fused_mc2=2).
• Paged attention and pa_shape_list.

Documentation

• Refreshed Ascend 950 DeepSeek-V3.1/V3.2 and GLM-5.2 deployment guidance. #12937† #13043†

Known Issues

• Pipeline parallelism (PP) combined with prefill context parallelism (PCP) is not supported in v0.23.0.
• Reduce sampling (additional_config.enable_reduce_sample=true) is experimental. It is incompatible with P/D-disaggregated serving and lmhead tensor parallelism, where it is automatically disabled. Do not enable it when sampling logprobs are requested because the logprob values and top-k rankings would be computed over partitioned logits and can be incorrect. #13469† #13632†
• GLM-5.2 1M-context deployments were validated only on Atlas 800 A3; the A2 series was not validated for 1M context.
• For P/D-disaggregated serving of models using Sparse Flash Attention (SFA), including DeepSeek-V3.2, GLM-5.1, and GLM-5.2, DCP must be enabled on both prefiller and decoder nodes or disabled on both. Asymmetric DCP configuration can cause accuracy issues. #14322†
• On Ascend 950, context parallelism is not supported with SFA; context-parallel MLA and GQA paths are experimental. #13303†
• In v0.23.0, combining GLM-5.2 DCP with Sparse Flash Attention C8 (enable_sparse_sfa_c8) has known issues, including performance degradation, and is not recommended. #14011†
• On Atlas 300I DUO or Atlas 200I Pro (310P), Qwen3.5-2B-W8A8 GSM8K accuracy can fluctuate by more than 1% compared with the floating-point model. #14335
• DeepSeek V4 on A2 or A3 can experience 1-2 second TPOT spikes under high concurrency when varying input shapes trigger Triton kernel recompilation. #14324
• DeepSeek V4 Flash and GLM-5.1 SWE benchmark results can vary because of intermittent evaluation failures, including container crashes, timeouts, dependency-download failures, and agent nondeterminism. #14326
• DeepSeek-V3.1 large-EP deployments have a reported 3%-5% performance gap from the target performance. #14327
• GLM-5.1 on an Ascend 950 single-node mixed P/D deployment can incur excessive P/D transfer overhead for a 64K-input/1K-output workload without a prefix-cache hit. #14328
• GLM-5.1 P/D-disaggregated serving with both sfa_c8 and li_c8 enabled has a reported performance regression for 64K-input/1K-output workloads. #14329
• In graph mode, torch.ops._C_ascend.mla_preprocess and torch.ops._C_ascend.batch_matmul_transpose can emit an rtMemcpy 107030 error in plog; the reported error does not affect inference. #14347
• GLM-5.2-W4A8C8 P/D-disaggregated serving with enable_sparse_sfa_c8 can show GPQA accuracy fluctuations of approximately 2-3 percentage points across repeated runs. #14378
• DeepSeek-V3.1 2P1D throughput has a reported regression of about 15% compared with v0.18.0 in high-throughput deployments. #12349
• DeepSeek V4 Pro has an open long-running stability report involving increasing memory use. #12345
• Qwen3-30B-A3B floating-point serving has a reported single-batch performance regression. #12337
• AscendStore KV Pool has backend-specific constraints for layerwise Memcache, SSD offload, hybrid-attention load failures, and graph fallback behavior. #12390
• Ascend 950 graph capture is temporarily limited to a reduced set of stages because of HDK incompatibility, which avoids crashes but can reduce performance. #12998