Skip to content

v1.3.0rc21

Pre-release
Pre-release

Choose a tag to compare

View all tags
@mikeiovine mikeiovine released this 15 Jul 22:46
v1.3.0rc21
1662a87
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

  • Deprecation Notices

    • AutoDeploy backend is being deprecated. We are aware that earlier model support is a critical priority for many users of TensorRT LLM and are working on agentic approaches to improve time to functional model support in the PyTorch backend. As an early indicator, this was used to release Minimax M3 functional support within the first week of model release.

  • Known Issues

    • DeepSeek V3.2
      • Host KV cache offload can fail on multi-GPU setups: executor init may hit GPU OOM (H200), or hang when combined with MTP.
      • Multi-GPU NVFP4 configs on B300 show accuracy failures.
    • Disaggregated Serving
      • DeepSeek V3 Lite + Helix (H100/H20): disaggregated generation can produce incorrect output vs. expected strings.
      • DeepSeek V3 Lite auto-dtype paths are failing in disaggregated mode, including:
      • auto_dtype_with_helix + CUDA graphs on B200/B300.
      • auto_dtype with MTP on RTX PRO 6000 Blackwell.
    • DeepSeek V3 / R1 Family (Aggregated PyTorch)
      • DeepSeek V3 Lite with torch.compile + CUDA graphs + overlap scheduler can hit CUDA launch failures during warmup on H100.
      • NVFP4 on RTX PRO 6000 Blackwell can OOM under tested configs.
      • Some multi-GPU parallelism combos (PP4, TP2PP2) with MTP + attention DP remain unstable or unvalidated; several are tracked under open NVBugs (e.g. 6428094, 6428096).
    • Llama 3.x
      • Llama 3.1 8B accuracy tests fail with torch.compile enabled (TRTLLM and FlashInfer backends) — same class of CUDA errors as DeepSeek V3 Lite.
      • Llama 3.1 FP8 PP4 with FlashInfer on GB300 is failing accuracy validation.
      • Llama 3.3 70B NVFP4 TP2PP2 (no torch.compile, no GEMM allreduce fusion) is failing.
    • GPT-OSS
      • W4 4-GPU serving configs (v1 KV cache + DP4/CUTLASS, EP4/TRTLLM FP8) are not passing accuracy checks.
      • EAGLE3 speculative decoding with VSWA reuse on H100 can fail with sampling errors during generation.
    • MiniMax M3
      • MXFP8 + piecewise CUDA graphs at TP8/EP8 on B300 fails due to a KV cache manager API mismatch (num_blocks_per_seq argument).
    • Qwen 3.5
      • An FP4 agg test on 8xB200 is crashing witb a tensor pointer alignment error.

  • Model Support

    • Add DeepSeek V4 (DSv4) model, tokenizer, and integration coverage (#15414)
    • Add Cosmos3 reasoner-only support and Cosmos3 audio output support (#15117, #14827)
    • Support Minimax M3 MXFP8 and NVFP4 checkpoints (#15687, #15857)
    • Add Gemma 4 12B Unified (encoder-free multimodal) support (#15768)
    • Add support for Qwen3.5-VL MoE and Dense variants (#14599, #15249)
    • Add Qwen3.6 NVFP4 checkpoint support (#15703)

  • API

    • BREAKING: Remove Python modules and tests for the legacy TensorRT backend (#15918)
    • BREAKING: Move multimodal-related args and env vars (#15640)
    • BREAKING: Rename server args (#16091)
    • BREAKING: Rename acceptance-rate-based dynamic speculation fields (#12905)
    • Add native /v1/embeddings dynamic batching for encoder-only models (#15424)
    • Add native post-processing hook to trtllm-serve (#15631)
    • Add prefix-aware scheduling config flag to support opt-out (#15526)

  • Feature

    • Add DSv4 sparse MLA attention backend (#15409)
    • Optimize DSv4 follow-ups: autotuner updates, disagg routing, DeepGEMM and MegaMoE, sparse attention and model defaults (#15626, #15625, #15632, #15717)
    • Support inflight weight update (#14815)
    • Expand dynamic speculation to all spec decode algorithms (#12262)
    • Enable CUDA graph execution for PyTorch encoder-decoder models (#15637)
    • Support Tensor Parallelism for PyTorch encoder-decoder models (#15897)
    • Add Piecewise CUDA Graph support for Qwen3.5/Qwen3.6 MoE models (#15900)
    • Add low-latency host task dispatch mode for guided decoding (#15863)
    • Dispatch GDN MTP target-verify to the FlashInfer bf16 kernel (#15975)
    • Optimize trtllm-gen MoE routing (#15656)
    • Allow fp8 per-tensor base weights for MoE LoRA (#15528)
    • Add an opt-in raw-weight cache to the HF weight loader (#16054)
    • Add per-model KV cache manager v2 auto selection (#15823)
    • Add multi-rank sleep/wakeup support to the MPI executor path (#14636)
    • Add proxy fast-death detection and sticky EngineDeadError (#15816)
    • Add a cascade attention implementation up to 4.6x faster than MMHA (#14396)
    • Add fused kernels for Gemma4 serving and improve Gemma4 inference correctness and performance (#16074, #15848)
    • Improve Qwen3-VL preprocessing performance and normalized model_config weight mapping (#15598, #16353)
    • Enable agent serving evaluation via trace-replay on Scaffolding (#14397)
    • Centralize sampling logic and split backends into isolated modules (#15542)
    • Add Cute DSL GVR Top-K short-row optimization that removes cluster sync in run_one_row (#15835)
    • Enable VSA in VisualGen (#14280)
    • Add Wan VAE backend (#15555)
    • Enable CUDA graph capture with torch.compile in VisualGen (#15603)
    • Add LTX-2 tile-parallel VAE decode and parallelize LTX-2 LoRA weight loading (#15753, #13911)
    • Cache LTX2 merged LoRA weight (#14984)
    • Fuse LTX-2 Gate + Residual + Norm + AdaLN modulation (ShiftScale) + Quant kernels (#15102)

  • Fix

    • Fix output distribution correctness for Eagle3 dynamic-tree rejection sampling (#15098)
    • Make EAGLE functional on the v1 KV cache manager (#15708)
    • Remove redundant residual bound check in EAGLE3 hidden-state capture (#16165)
    • Correct speculative XQA attention sinks (#15739)
    • Add spec_metadata=None kwarg to SpecWorkerBase._apply_force_accepted_tokens (#15797)
    • Gate the py_last_draft_tokens snapshot in PyExecutor._prepare_draft_requests (#16121)
    • Fix bugs in Beam Search kernels (#15621)
    • Fix logits post processor for beam search in the PyTorch backend (#16010)
    • Prevent disaggregated KV transfer stalls and fix flaky hang for disagg gen-only (#15737, #16172)
    • Enable disagg partial reuse store for PP>1 (#15655)
    • Fix indexer-k-cache transfer for disagg (#16197)
    • Make cache transceiver transport reporting deterministic (#15893)
    • Pass dtype to AllReduce ctor to enable MNNVL all-reduce (#15547)
    • Use persistent per-stream workspace in cublas_mm for CUDA-graph safety (#15534)
    • Fix FMHA kernels not found for GPT-OSS + SM120 (#15230)
    • Add a runtime guard in FlashInferTrtllmGenAttention.is_supported (#15496)
    • Fix RoPE support in the flashinfer trtllm-gen backend (#15661)
    • Update trtllm-gen FMHA JIT libraries (#15974)
    • Fix TRTLLM-GEN backend multiCtasKv counter clear (#15761)
    • Sync CTA before PDL trigger in quantize_with_block_size (#14668)
    • Add EP assertion to DenseGEMMFusedMoE (#15451)
    • Cap MXFP4 Hopper swizzle transient GPU memory during MoE weight load (#16125)
    • Restrict FP8 rowwise handling for compressed-tensors: resolve config group by name and flatten per-channel weight_scale (#15415)
    • Fix GLM-5.1 NVFP4 fallback to AR-Norm fusion for unquantized dense layers (#15659)
    • Restrict MiniMax M3 dense SDPA backends and enable MiniMax M3 piecewise CUDA graphs (#15937, #15923)
    • Fix NT3 NVFP4 perf regression on Blackwell (#16031)
    • Fix Qwen2-VL Transformers 5 compatibility (#15997)
    • Fix Gemma4 MoE weight loading (#16108)
    • Honor Qwen-Image quant ignore list (#15599)
    • Fix MLA KV cache estimation sizing (#16311)
    • Size DSV4 KV constraint with num_extra_kv_tokens (#16173)
    • Source DSA metadata from sparse params and add a persistent topk-output buffer to avoid CUDA-graph stale-pointer IMA (#16236, #16290)
    • Fix fused mHC output reuse and extend compressor next_n (#16221)
    • Keep SSM cache in weights dtype when materializing (#16065)
    • Correct RocketKV KT cache byte accounting for FP8 KV cache (#16225)
    • Fix VSWA gate in KVCacheManager window-size resolution (#16233)
    • Reserve worst-case SWA slots to avoid single-request deadlock (#15588)
    • Add seq-slot pool overlap headroom with consistent slot-indexed buffer sizing (#16279)
    • Reduce per-request attribute overhead in the model_engine hot path (#15750)

  • Documentation

    • Remove legacy TensorRT docs and add a migration guide (#15767)
    • Add VisualGen engineering criteria (#15225)
    • Add VisualGen MGMN NVL72 tech blog and NVL72 video generation blog to README (#15420, #15858)
    • Clarify dtype='auto' resolution for LLM and KvCacheConfig (#15520)

  • Benchmark

    • Add in-process NeMo-Skills accuracy benchmarks (#15608)
    • Add Kimi-K2.5 disaggregated GSM8K accuracy test (#15617)
    • Add gpt-oss-120b eagle3 accuracy test (#14990)

  • Test & Infra

    • Upgrade dependencies for the dlfw 26.04 stack (#12643)
    • Upgrade NIXL to v1.3.0, aiperf to 0.8.0, ray to 2.55.1, and flashinfer-python to 0.6.14 (#15694, #15769, #15819, #15869)
    • Make FlashInfer a hard dependency for the Torch sampler (#16160)
    • Remove legacy TensorRT examples, TensorRT-backend tests, and the legacy TensorRT-engine Triton backend (#15763, #15810, #15907)
    • Retire Triton backend TRT workflow QA tests (#15814)
    • Add a model-derived PyTorch attention backend test suite (#15536)
    • Add test coverage for Eagle3ForCausalLM.apply_eagle3_fc (#16107)
    • Add KV cache manager v2 (V2 + VSWA) multi-GPU test coverage (#16114)
    • Add DSv4 coverage and import safety checks (#15710)
    • Add gemma and glm disagg python transceiver tests (#15755)
    • Add a CPU-only CI stage (part 1) (#15815)

What's Changed

  • [https://nvbugs/6301807][fix] Fix FP8 rowwise linear reference precision by @DylanChen-NV in #15421
  • [None][feat] DSv4: sparse MLA attention backend by @lfr-0531 in #15409
  • [None][feat] Converge VisualGen LPIPS and VBench test generation by @yibinl-nvidia in #14654
  • [TRTLLM-12721][fix] Bound disagg transfer polling and admission by @chienchunhung in #15356
  • [None][feat] Support inflight weight update by @shuyixiong in #14815
  • [None][fix] Fall back to NVLink P2P when NVLS fabric is unprovisioned by @Wanli-Jiang in #15302
  • [TRTLLM-13629][test] Optimize MoE CI test-db by @xxi-nv in #15624
  • [None][feat] DSv4: model, tokenizer, and integration coverage by @lfr-0531 in #15414
  • [TRTLLM-13265][feat] Fuse LTX-2 Gate + Residual + Norm + AdaLN modulation(ShiftScale) + Quant kernels by @luyiyun1021 in #15102
  • [TRTLLM-11715][infra] Upgrade dependencies for dlfw 26.04 stack by @EmmaQiaoCh in #12643
  • [None][fix] GLM-5.1 NVFP4 fallback to AR-Norm fusion for unquantized dense layers by @syuoni in #15659
  • [None][perf] DSv4 follow-up: autotuner updates by @lfr-0531 in #15626
  • [None][chore] split unittest/_torch/visual_gen by @tburt-nv in #15670
  • [None][chore] Decrease the qwen3.5 ci test time by @JadoTu in #15646
  • [None][test] Waive 1 failed cases for main in QA CI by @tensorrt-cicd in #15678
  • [None][feat] Cache LTX2 merged LoRA weight by @yibinl-nvidia in #14984
  • [None][test] Waive 1 failed cases for main in QA CI by @tensorrt-cicd in #15688
  • [TRTLLM-12751][feat] visual-gen /metrics iteration stats producer by @JunyiXu-nv in #14246
  • [None][infra] Waive 9 failed cases for main in post-merge 2810 by @ZhanruiSunCh in #15700
  • [None][test] record per-case hostname in perf result CSV by @ruodil in #15663
  • [None][test] Waive 2 failed cases for main in QA CI by @tensorrt-cicd in #15667
  • [None][test] Waive 4 failed cases for main in QA CI by @tensorrt-cicd in #15686
  • [None][infra] Waive 5 failed cases for main in post-merge 2811 by @ZhanruiSunCh in #15705
  • [None][feat] Optimize trtllmgen moe routing by @jiahanc in #15656
  • [None][feat] add CI-enforced stability gate for trtllm-serve HTTP schema by @JunyiXu-nv in #15644
  • [None][feat] Bind NUMA-aware CPU affinity in visual_gen workers by @luyiyun1021 in #13590
  • [TRTLLM-13319][fix] fix output distribution correctness for Eagle3 dynamic-tree rejection sampling by @zhaoyangwang-nvidia in #15098
  • [None][test] Waive 4 failed cases for main in post-merge by @tensorrt-cicd in #15721
  • [None][test] Waive 1 failed cases for main in QA CI by @tensorrt-cicd in #15719
  • [None][ci] Disable home mount for sbatch L0 tests by @chzblych in #15713
  • [None][feat] Parallelize host KV cache pool prefault and add THP control by @nafis271 in #15431
  • [TRTLLM-13613][test] Trim duplicated and dead multimodal accuracy tests from pre-merge CI by @Wanli-Jiang in #15615
  • [None][test] AutoDeploy: Fix standalone linear simple on B200 by @govind-ramnarayan in #15675
  • [None][test] Enable single-GPU LPIPS CI protection for VisualGen models by @chang-l in #15564
  • [https://nvbugs/6287834][chore] Unwaive GPT-OSS disagg perf sanity case by @peihu-nv in #15600
  • [https://nvbugs/6273845][fix] Fix FMHA kernels are not found for GPTOSS + SM120 by @tensorrt-cicd in #15230
  • [https://nvbugs/6316980][fix] Added a runtime guard in FlashInferTrtllmGenAttention.is_supported using the… by @tensorrt-cicd in #15496
  • [TRTLLM-13543][feat] WideEP FT: add EPLB mask-only reconfigure (1b.1 + 1b.2) by @chienchunhung in #15525
  • [None][docs] Add VisualGen engineering criteria by @zhenhuaw-me in #15225
  • [None][feat] Add 'kimi_k2.5_fp4' to TRUST_REMOTE_CODE_MODELS in performance tests by @yufeiwu-nv in #15751
  • [None][chore] Bump version to 1.3.0rc21 by @yuanjingx87 in #15742
  • [None][test] Waive 8 failed cases for main in QA CI by @tensorrt-cicd in #15733
  • [None][test] Waive hang issues by @xinhe-nv in #15764
  • [None][test] remove two model tests from test list by @xinhe-nv in #15701
  • [None][feat] kv cache manager v1 use fabric memory to enable mnnvl transfer for kv cache transfer in disaggregated serving by @chuangz0 in #12490
  • [None][test] Waive 8 failed cases for main in post-merge by @tensorrt-cicd in #15774
  • [None][infra] Waive 5 failed cases for main in pre-merge 45198 by @ZhanruiSunCh in #15779
  • [None][test] waive tests by @xinhe-nv in #15778
  • [TRTLLM-13580][test] Add model-derived PyTorch attention backend test suite by @yuxianq in #15536
  • [https://nvbugs/6150288][fix] Use persistent per-stream workspace in cublas_mm for CUDA-graph safety by @pamelap-nvidia in #15534
  • [None][feat] Cosmos3 reasoner only support by @bastefaniak in #15117
  • [None][chore] Autodeploy disable the pipeline cache by default by @nvchenghaoz in #15530
  • [https://nvbugs/6266306][fix] Fix testing harness by @SimengLiu-nv in #15558
  • [None][feat] Sharding IR default (auto-detect) + Eagle/MTP draft sharding + qwen3_next by @greg-kwasniewski1 in #14786
  • [None][perf] set ncclConfig graphUsageMode=1 on communicator init by @nv-lschneider in #13511
  • [None][feat] VisualGen: enable CUDA graph capture with torch.compile by @chang-l in #15603
  • [None][fix] Honor Qwen Image quant ignore list by @pst2154 in #15599
  • [https://nvbugs/6250439][fix] Correct speculative XQA attention sinks by @dongfengy in #15739
  • [https://nvbugs/6336747][ci] Unwaive nemotron nvfp4 E2E test by @2ez4bz in #15561
  • [TRTLLM-12622][feat] Reland: Add native post-processing hook to trtllm-serve by @xwang233 in #15631
  • [TRTLLM-12200][feat] WideEP FT: add active_rank_mask to NVLink AlltoAll kernels (1a.2) by @chienchunhung in #13404
  • [None][chore] Allow fp8 per-tensor base weights for MoE LoRA by @brb-nv in #15528
  • [https://nvbugs/6331421][fix] Fix TRTLLM-GEN backend multiCtasKv counter clear by @pengbowang-nv in #15761
  • [None][feat] Add prefix-aware scheduling config flag to support opt-out by @SimengLiu-nv in #15526
  • [None][feat] Handle empty-block ranks with Helix CP by @brb-nv in #15383
  • [https://nvbugs/6162506][test] AutoDeploy: fix NVFP4 MoE fp4 weight scale in test by @govind-ramnarayan in #15748
  • [TRTLLM-13630][test] Reuse MPIPoolExecutor across MoE multi-GPU test cases by @xxi-nv in #15744
  • [https://nvbugs/6196614][infra] Upgrade aiperf to 0.8.0 by @EmmaQiaoCh in #15769
  • [TRTLLM-13409][feat] hard-exit on HangDetector fire + cross-rank propagation by @JunyiXu-nv in #15612
  • [None][test] Waive 1 failed cases for main in post-merge by @tensorrt-cicd in #15811
  • [None][feat] Support update weight for nemotron-h by @shuyixiong in #15573
  • [None][fix] compressed-tensors fp8: resolve config group by name and flatten per-channel weight_scale by @Hudayday in #15415
  • [TRTLLM-8997][feat] Add encoder_max_batch_size & encoder_max_num_tokens and deterministic dummy sizing by @yechank-nvidia in #13503
  • [None][feat] Support DSA cross-layer indexer top-k sharing (e.g. GLM-5.2) by @dc3671 in #15574
  • [None][feat] add CI-enforced stability gate for trtllm-serve CLI options by @JunyiXu-nv in #15643
  • [None][infra] Waive 10 failed cases for main in post-merge 2814 by @trtllm-agent in #15825
  • [TRTLLM-13168][feat] test best ucx env for cache transceiver by @chuangz0 in #14933
  • [https://nvbugs/6342840][fix] Add spec_metadata=None kwarg to SpecWorkerBase._apply_force_accepted_tokens… by @tensorrt-cicd in #15797
  • [None][perf] Eliminate torch.where host sync in multimodal fuse_input_embeds path by @yechank-nvidia in #14384
  • [https://nvbugs/5955773][test] Unwaive test_no_kv_cache_reuse for DeepSeekV3Lite by @kaiyux in #15826
  • [None][infra] Upgrade NIXL to v1.3.0 by @zackyoray in #15694
  • [None][infra] Waive 1 failed cases for main in pre-merge 45709 by @trtllm-agent in #15843
  • [None][infra] Waive 1 failed cases for main in pre-merge 45709 by @trtllm-agent in #15845
  • [None][infra] Waive 1 failed cases for main in pre-merge 45709 by @trtllm-agent in #15847
  • [TRTLLM-12955][fix] use get_free_port_in_ci to avoid binding to a port in use by @NVShreyas in #15786
  • [None][doc] add VisualGen MGMN NVL72 tech blog by @NVShreyas in #15420
  • [None][fix] Add AutoDeploy post-merge stages by @hnover-nv in #15418
  • [https://nvbugs/6266302][test] Unwaive test_disaggregated_llama_context_capacity by @Tabrizian in #15800
  • [None][chore] Waive failing allreduce tests by @brb-nv in #15850
  • [TRTLLM-13458][feat] Support Minimax M3 MXFP8 checkpoint by @WeiHaocheng in #15687
  • [None][chore] Simplify MoE lora by @brb-nv in #15330
  • [None][fix] report NCCL init timeout by @yuxianq in #15649
  • [TRTLLM-13782][doc] Remove legacy TensorRT docs and add migration guide by @Wanli-Jiang in #15767
  • [None][infra] Waive 1 failed cases for main in pre-merge 45748 by @trtllm-agent in #15862
  • [https://nvbugs/6342840][infra] Un-waive 37 mtp perf-sanity cases (18 fixed by #15797 + 19 for CI recheck) by @chenfeiz0326 in #15827
  • [TRTLLM-12823][ci] Drop duplicate test by @YihuiLu512 in #15396
  • [TRTLLMINF-152][infra] Waive tests that have too many logs in system-out by @yiqingy0 in #15749
  • [None][fix] handle rank-3 Fp4 and non-contiguous Ulysses shard in LTX-2 NVFP4 fused paths by @luyiyun1021 in #15718
  • [None][doc] add NVL72 video generation blog to README by @chang-l in #15858
  • [None][test] Waive 9 failed cases for main in QA CI by @trtllm-agent in #15867
  • [None][infra] Waive 2 failed cases for main in post-merge 2815 by @trtllm-agent in #15870
  • [None][perf] DSv4 follow-up: disagg routing improvements by @lfr-0531 in #15625
  • [TRTLLM-13685][chore] Move MLA module to separate file by @yuxianq in #15648
  • [NVBUG-6266259][fix] Fix userbuffers prologue patterns by @liji-nv in #15220
  • [TRTLLM-13120][feat] Cosmos3 Audio Output Support by @NVShreyas in #14827
  • [TRTLLM-13781][chore] Remove legacy TensorRT examples by @Wanli-Jiang in #15763
  • [https://nvbugs/6385673][fix] Add isinstance guards for parameters, properties, and the per-parameter… by @trtllm-agent in #15813
  • [https://nvbugs/6385283][fix] Add empty/absent-content guard in the "message" and "reasoning" branches that… by @trtllm-agent in #15812
  • [https://nvbugs/6376948][fix] Reduce per-request attribute overhead in model_engine hot path by @chenfeiz0326 in #15750
  • [None][test] Add Kimi-K2.5 disaggregated GSM8K accuracy test by @Shixiaowei02 in #15617
  • [None][infra] Waive 1 failed cases for main in pre-merge 45955 by @trtllm-agent in #15891
  • [#14225][perf] AutoDeploy MTP + ADP enablement and MoE all-to-all optimization by @MrGeva in #15063
  • [TRTLLM-13546][feat] Add error classification patterns (1c.1) by @chienchunhung in #15677
  • [None][infra] Waive 1 failed cases for main in pre-merge 45992 by @trtllm-agent in #15894
  • [None][infra] Waive 2 failed cases for main in pre-merge 45992 by @trtllm-agent in #15898
  • [None][perf] Cascade attention impl faster than MMHA 4.6x by @Fallenpetal in #14396
  • [None][infra] Improve unit test CI coverage by @yuxianq in #15368
  • [None][test] Waive 4 failed cases for main in QA CI by @trtllm-agent in #15902
  • [None][infra] Waive 2 failed cases for main in pre-merge 45969 by @trtllm-agent in #15901
  • [https://nvbugs/6410963][chore] Waive a failed test in Pre-merge by @shuyixiong in #15904
  • [https://nvbugs/6095851][fix] Refresh FlashMLA metadata for draft KV cache by @yizhang-nv in #15251
  • [None][test] reuse conftest mpi_pool_executor in MoE routing multi-GPU test by @xxi-nv in #15868
  • [https://nvbugs/6208457][fix] Pass IPC HMAC key through file descriptor by @yibinl-nvidia in #15654
  • [TRTLLM-13446][test] Add FLUX infer() num_images_per_prompt forwardin… by @yingguo-trt in #15745
  • [None][chore] Upgrade ray version to 2.55.1 by @shuyixiong in #15819
  • [TRTLLM-13947][test] Retire Triton backend TRT workflow QA tests by @StanleySun639 in #15814
  • [None][infra] Waive 12 failed cases for main in post-merge 2816 by @trtllm-agent in #15914
  • [TRTLLM-13445][test] Add SageAttention backend-routing and SAGE recip… by @yingguo-trt in #15746
  • [TRTLLM-13581][test] Add moe fp4 gpu dequant and unpack optimization by @rosong11 in #15714
  • [NVBUG-6280721][fix] Unwaive DeepSeek-V3.2 FP4 MTP perf-sanity test by @xguannv in #15783
  • [TRTLLM-12950][perf] DSv4 follow-up: DeepGEMM and MegaMoE by @lfr-0531 in #15632
  • [https://nvbugs/6394425][fix] Added _is_effective_dynamic_tree() helper in utils.py that returns True only… by @trtllm-agent in #15842
  • [None][feat] DSv4 follow-up: runtime KV and cache foundations by @jiaganc in #15633
  • [None][feat] Qwen-Image: load pre-quantized ModelOpt NVFP4/FP8 checkpoints by @jingyu-ml in #15470
  • [TRTLLM-13458][feat] Support Minimax M3 NVFP4 checkpoint by @pcicotti in #15857
  • [None][feat] Add Gemma 4 12B Unified (encoder-free multimodal) support by @Hudayday in #15768
  • [None][perf] overlap LTX-2 v2a cross-attn comm with compute via async-Ulysses by @luyiyun1021 in #15754
  • [TRTLLM-14040][perf] VisualGen: Eliminate worker-server decoded media IPC by using shared-tensor handle by @luyiyun1021 in #15834
  • [None][test] Waive 1 failed cases for main in QA CI by @trtllm-agent in #15930
  • [TRTLLM-11556][feat] Expand dynamic speculation to all spec decode algorithms by @zheyuf in #12262
  • [https://nvbugs/6402048][test] Skip MXFP8 CUTLASS MoE with non-128-aligned per-shard intermediate by @xxi-nv in #15865
  • [https://nvbugs/6398200][fix] Remove fixture-level params on mpi_pool_executor to avoid duplicate parametrization by @xxi-nv in #15860
  • [TRTLLM-12500][feat] Add support for Qwen3.5 VL MoE (with the MTP fixes) by @moraxu in #14599
  • [TRTLLM-13176][feat] Enables CUDA graph execution for PyTorch encoder-decoder models by @cascade812 in #15637
  • [https://nvbugs/6378901][fix] Defer speculative D2H sync-check hooks until after LLM warmup by @brb-nv in #15796
  • [None][ci] Use x86 daemon host in Slurm jobs everywhere by @chzblych in #15942
  • [None][infra] Ensure Slurm cleanup runs after interruptions by @chzblych in #15943
  • [None][ci] Use x86 image for daemon host in Slurm jobs everywhere by @chzblych in #15948
  • [https://nvbugs/6401921][fix] Stabilize single-GPU Wan2.2/LTX2/Wan2.1 LPIPS test by @chang-l in #15854
  • [None][infra] Waive 1 failed cases for main in pre-merge 46352 by @trtllm-agent in #15949
  • [https://nvbugs/6385256][fix] Guard at both layers — add len>0 checks in get_request_num_tokens, return []… by @tensorrt-cicd in #15781
  • [TRTLLM-13973][fix] Restrict MiniMax M3 dense SDPA backends by @brb-nv in #15937
  • [https://nvbugs/6405665][test] Disable block reuse for KV cache comparison by @jiaganc in #15917
  • [https://nvbugs/6344107][fix] Enable disagg partial reuse store for PP>1 by @nv-xtf in #15655
  • [None][infra] Waive 1 failed cases for main in pre-merge 46289 by @trtllm-agent in #15946
  • [None][fix] Ignore valid Faroese code in codespell by @jiaganc in #15954
  • [None][chore] Update flashinfer-python from 0.6.12 to 0.6.14 by @yihwang-nv in #15869
  • [https://nvbugs/6384951][fix] Fix incorrect error reporting by @2ez4bz in #15732
  • [https://nvbugs/6337231][fix] Unskip and fix iter-stats unit tests' fake-self predicate by @YihuiLu512 in #15927
  • [#14588][feat] AutoDeploy: DeepSeek-R1 optimization for low concurrency by @taylor-yb-lee in #14810
  • [None][test] Remove 72 closed-bug waive entries for main by @trtllm-agent in #15959
  • [None][fix] Pass dtype to AllReduce ctor to enable MNNVL all-reduce fo… by @nv-guomingz in #15547
  • [None][perf] DSv4 follow-up: sparse attention and model defaults by @lfr-0531 in #15717
  • [https://nvbugs/6401925][fix] Unwaive DeepSeek V4 compressor corner-case test by @yuxianq in #15957
  • [TRTLLM-13628][test] Optimize MoE comm test execution by @sunnyqgg in #15662
  • [https://nvbugs/6396420][fix] Restore single-node NVLS over POSIX-FD by @Wanli-Jiang in #15882
  • [None][fix] Remove unreachable model path from MTPDraftModel by @nv-guomingz in #15722
  • [None][test] Add gpt-oss-120b eagle3 accuracy test by @JennyLiu-nv in #14990
  • [None][infra] Fix KeyError for test names with spaces in generate_rerun_tests_list by @yiqingy0 in #15770
  • [None][fix] Fix marlin_nvfp4_template.h compilation error by @yihwang-nv in #15660
  • [https://nvbugs/6248837][fix] Free worker GPU memory on executor shutdown by @tianyuxbear in #15099
  • [https://nvbugs/6341070][fix] Fix scaffolding MajorityVoteController output handling by @KleinBlueC in #15925
  • [None][infra] Waive 11 failed cases for main in post-merge 2822 by @trtllm-agent in #15960
  • [TRTLLM-13334][fix] Add EP assertion to DenseGEMMFusedMoE by @guqiqi in #15451
  • [https://nvbugs/6384622][fix] increase free_mem_frac for failed perf test by @reasonsolo in #15879
  • [None][test] Waive 1 failed cases for main in QA CI by @trtllm-agent in #15962
  • [None][infra] Waive 50 failed cases for main in post-merge 2822 by @trtllm-agent in #15967
  • [None][infra] Coerce null CBTS PR diffs to empty string by @crazydemo in #15874
  • [None][infra] Waive 5 failed cases for main in post-merge 2823 by @trtllm-agent in #15968
  • [TRTLLMINF-126][infra] Generate timeout result when the tests all pass in the rerun step by @yiqingy0 in #15658
  • [None][test] Increase timeout for Kimi-K2.5 disaggregated nvfp4 test by @Shixiaowei02 in #15970
  • [https://nvbugs/6299530][fix] Capture Qwen3.5 GDN for piecewise CUDA … by @liji-nv in #15594
  • [None][test] Waive 1 failed cases for main in QA CI by @trtllm-agent in #15977
  • [TRTLLM-13584][feat] add Wan VAE backend by @taianz-nv in #15555
  • [TRTLLM-13783][test] Remove TensorRT-backend tests by @Wanli-Jiang in #15810
  • [https://nvbugs/6388787][fix] Revert Pass IPC HMAC key through file descriptor (#15654) by @chenfeiz0326 in #15961
  • [https://nvbugs/6337229, https://nvbugs/6341072][fix] Stabilize scaffolding MCP worker test server startup and cleanup by @KleinBlueC in #15958
  • [None][perf] Improve inference correctness and perf for Gemma4 by @2ez4bz in #15848
  • [None][infra] Bump pulse-container-scanner version by @yuanjingx87 in #15856
  • [None][infra] PLC nightly pipeline update by @yuanjingx87 in #15274
  • [TRTLLM-12647][feat] Parallelize LTX-2 LoRA weight loading by @yibinl-nvidia in #13911
  • [TRTLLM-13248][feat] Wave 3: migrate MoE staged hooks by @chienchunhung in #15386
  • [TRTLLM-12557][feat] WideEP FT: add AlltoAll watchdog (1a.3 + 1a.4) by @chienchunhung in #15524
  • [https://nvbugs/6422094][fix] Waive failing AutoDeploy UT when testing PR#15923 by @longlee0622 in #16023
  • [None][feat] add gemma and glm disagg python transceiver test by @chuangz0 in #15755
  • [TRTLLM-13968][fix] Enable MiniMax M3 piecewise CUDA graphs by @liji-nv in #15923
  • [None][fix] fix bench-moe benchmark bugs by @guqiqi in #15397
  • [None][infra] Waive 16 failed cases for main in post-merge 2824 by @trtllm-agent in #16027
  • [None][perf] Move greedy stop checks to host by @mingyangHao in #15920
  • [https://nvbugs/6336801][fix] Fix attention op sync test by @yuxianq in #15956
  • [https://nvbugs/6114139][test] Unwaive test_workers_kv_cache_events by @eopXD in #15979
  • [None][test] Waive 8 failed cases for main in QA CI by @trtllm-agent in #16030
  • [None][test] Waive 7 failed cases for main in QA CI by @trtllm-agent in #16034
  • [None][infra] Add blossom-ci authorized users by @yiqingy0 in #16029
  • [TRTLLM-14029][feat] LTX-2 tile-parallel VAE decode by @luyiyun1021 in #15753
  • [None][refactor] Refine Skip Softmax follow-ups by @bobboli in #15417
  • [None][feat] Disaggregated KV-cache bounce transfer by @Shixiaowei02 in #15618
  • [https://nvbugs/6303211][fix] set 2 multi_round to save time by @bo-nv in #16043
  • [TRTLLM-13249][feat] Wave 4: add MX staged receiver cutover by @chienchunhung in #15387
  • [None][infra] Add blossom-ci authorized users by @yiqingy0 in #16046
  • [None][feat] Dispatch GDN MTP target-verify to FlashInfer bf16 kernel by @nv-guomingz in #15975
  • [None][infra] waive failing tests in Post Merge by @mzweilz in #16039
  • [None][test] keep MXFP4 block-scale unswizzle on GPU in create_weights by @xxi-nv in #15951
  • [None][infra] add disable-cbts flag to explicitly disable CBTS by @crazydemo in #15830
  • [https://nvbugs/6321874][fix] Filter .lock from os.listdir(...) in the test's rank-file count; skip the… by @trtllm-agent in #15866
  • [https://nvbugs/6242591][fix] Fix bugs in Beam Search kernels by @wili-65535 in #15621
  • [None][test] Waive 4 failed cases for main in QA CI by @trtllm-agent in #16032
  • [None][test] Waive 9 failed cases for main in QA CI by @trtllm-agent in #16058
  • [None][perf] Validate GPT-OSS transceiver v2 performance by @nv-xtf in #15765
  • [TRTLLM-12154][test] Add mixed-stress disagg client and pytest variants for Qwen3-32B FP8 by @brnguyen2 in #16008
  • [TRTLLM-12347][feat] enable VSA in VisualGen by @o-stoner in #14280
  • [https://nvbugs/6403909][fix] unwaive an attention backend test by @yuxianq in #15964
  • [TRTLLM-10253][infra] upload test output to storage by @niukuo in #13073
  • [https://nvbugs/6410963][fix] Make cache transceiver transport reporting deterministic by @chienchunhung in #15893
  • [None][infra] Waive 1 failed cases for main in pre-merge 46573 by @trtllm-agent in #16082
  • [TRTLLM-13352][chore] BREAKING: move multimodal related args / env vars by @2ez4bz in #15640
  • [None][fix] expose startup KV cache capacity by @tedzhouhk in #15385
  • [None][build] Use cp -rL and skip up-to-date copies in copy_resolving_symlink by @brnguyen2 in #16020
  • [None][build] Add --yes flag to build_wheel.py to skip interactive prompt by @brnguyen2 in #16015
  • Revert "[https://nvbugs/6403909][fix] unwaive an attention backend test" by @tburt-nv in #16085
  • [None][perf] Improve Qwen3-VL Preprocessing Perf by @aswinvisva in #15598
  • [https://nvbugs/6337235][test] Remove stale MX/GMS model loader skips by @chienchunhung in #15987
  • [TRTLLM-13383][feat] Add support for Qwen3.5 VL Dense by @moraxu in #15249
  • [None][fix] Fix Qwen2-VL Transformers 5 compatibility by @2ez4bz in #15997
  • [None][infra] Waive 10 failed cases for main in post-merge 2825 by @trtllm-agent in #16093
  • [None][doc] Clarify dtype='auto' resolution for LLM and KvCacheConfig by @ojas4414 in #15520
  • [None][fix] Unwaive supported attention backend cases by @yuxianq in #15838
  • [https://nvbugs/6236818][ci] Re-enable Verl post-merge CI stage by @Superjomn in #15935
  • [None][test] Waive 2 failed cases for main in QA CI by @trtllm-agent in #16097
  • [None][infra] Waive 1 failed cases for main in pre-merge 46666 by @nv-guomingz in #16100
  • [https://nvbugs/6427411][chore] Waive failed tests in Pre-merge by @shuyixiong in #16103
  • [None][test] Modularized attention perf round 2: per-case isolation, device-name keying, GQA/MLA/DSA cases by @ruodil in #15899
  • [https://nvbugs/6427411][chore] Waive a failed test in Pre-merge by @shuyixiong in #16105
  • [https://nvbugs/6336747][ci] Waive intermittently hanging E2E tests by @2ez4bz in #16081
  • [https://nvbugs/6422332][fix] Keep SSM cache in weights dtype when ma… by @nv-guomingz in #16065
  • [None][opt] Dsv4-Pro attn kernel epilogue fuse RopeQuant (#14947) by @yunruis in #16036
  • [None][fix] Fix logits post processor for beam search in PyTorch backend by @cascade812 in #16010
  • [None][infra] Optionally redirect AutoDeploy imports to llm-c by @bmarimuthu-nv in #15726
  • [None][infra] Waive 23 failed cases for main in post-merge 2826 by @trtllm-agent in #16109
  • [https://nvbugs/6396728][fix] Add module-level pytestmark = pytest.mark.threadleak(enabled=False) to… by @trtllm-agent in #15839
  • [https://nvbugs/6388359][test] Unwaive Qwen3-30B-A3B test_w4a8_mxfp4 mxfp8-latency-CUTLASS by @xxi-nv in #16110
  • [None][ci] relocate test_qwen_moe_routed_expert_multi_lora_varying_ranks by @QiJune in #16118
  • [None][feat] Add low-latency host task dispatch mode for guided decoding by @achartier in #15863
  • [TRTLLM-13212][refactor] Centralize sampling logic, split backends into isolated modules by @zhaoyangwang-nvidia in #15542
  • [None][infra] Remove attention op sync test waivers by @yuxianq in #16117
  • [https://nvbugs/6427411][chore] Waive remaining PP sampler regressions by @chienchunhung in #16127
  • [TRTLLM-13175][feat] Support Tensor Parallelism PyTorch encoder-decoder models by @cascade812 in #15897
  • [TRTLLM-13250][feat] Wave 5: Enable MX post-transform Llama receiver by @chienchunhung in #15432
  • [https://nvbugs/6427411][chore] Waive additional PP sampler regressions by @chienchunhung in #16146
  • [None][chore] Enable acc for disagg multi node stress test on qa side by @fredricz-20070104 in #16040
  • [https://nvbugs/6422299][fix] Setup dependency for Cosmos3 nano t2v LPIPS test by @yibinl-nvidia in #16139
  • [None][fix] visual_gen FLUX: enable fused DiT QK-norm + RoPE by default by @chang-l in #15676
  • [TRTLLM-13709][feat] Add the Qwen3.6 nvfp4 checkpoint support by @nv-guomingz in #15703
  • [https://nvbugs/6103083][fix] Make EAGLE functional on v1 KV cache manager by @eopXD in #15708
  • [https://nvbugs/6316983][fix] Fix RoPE support in flashinfer trtllm-gen backend by @yihwang-nv in #15661
  • [None][perf] Port remaining DeepSeek V4 optimizations to main by @lfr-0531 in #16028
  • [None][infra] Automate and gate LLM args telemetry manifest updates by @venkywonka in #16013
  • [None][infra] Waive 4 failed cases for main in post-merge 2827 by @trtllm-agent in #16169
  • [DEP-950][feat] add multi-rank sleep/wakeup support to MPI executor path by @hhzhang16 in #14636
  • [TRTLLM-13977][fix] Fix NT3 NVFP4 perf regression in Blackwell by @xuantengh in #16031
  • [None][docs] Default GitHub CLI config directory non-interactively by @bobboli in #15978
  • [None][test] DSv4 PR-6 coverage and import safety by @lfr-0531 in #15710
  • [TRTLLM-14155][fix] Revert host-side greedy stop checks from #15920 by @mingyangHao in #16163
  • [TRTLLMINF-144][infra] Only display rerun test results in the rerun report by @yiqingy0 in #15619
  • [TRTLLM-13784][chore] Remove legacy TensorRT-engine Triton backend by @Wanli-Jiang in #15907
  • [None][test] Remove H200 and RTX6000-D for QA perf Test by @yufeiwu-nv in #16183
  • [None][test] Reduce Nemotron V3 Super/Ultra pre-merge test cases by @nv-guomingz in #16122
  • [None][infra] Waive 2 failed cases for main in pre-merge 46981 by @trtllm-agent in #16186
  • [None][test] Remove llm_function_l20.txt test list by @jieli-matrix in #16187
  • [None][test] copy AD smoke tests to LLMC standalone by @bmarimuthu-nv in #16138
  • [TRTLLM-14080][test] Add test coverage for Eagle3ForCausalLM.apply_eagle3_fc by @StanleySun639 in #16107
  • [https://nvbugs/6390307][fix] Update trtllm-gen FMHA JIT libraries by @bobboli in #15974
  • [TRTLLMINF-81][fix] Scope SLURM retry node-avoidance to the target cluster by @dpitman-nvda in #15992
  • [None][fix] Retry SLURM dispatcher pod launch failures by @dpitman-nvda in #16079
  • [TRTLLMINF-94][infra] Toggle ENABLE_BOLT_COMPATIBLE by @mlefeb01 in #15428
  • [None][fix] Remove redundant residual bound check in EAGLE3 hidden-state capture by @eopXD in #16165
  • [https://nvbugs/6418912][fix] Fix KV-cache lifecycle test failures by @chienchunhung in #16019
  • [https://nvbugs/6418103][fix] Clamp the post-allreduce quota by the pre-allreduce quota (`quota = min(quota… by @trtllm-agent in #15991
  • [None][ci] Update golden telemetry manifest by @2ez4bz in #16208
  • [https://nvbugs/6418090][fix] Add @torch.inference_mode() decorator to… by @trtllm-agent in #16000
  • [TRTLLM-13406][feat] add in-process NeMo-Skills accuracy benchmarks by @Wanli-Jiang in #15608
  • [None][chore] Centralize trtllm-gen FMHA unsupported diagnostics by @yihwang-nv in #16104
  • [None][test] Fix MLA import path in attention perf harness by @ruodil in #16179
  • [None][infra] Add blossom-ci authorized users by @yiqingy0 in #16210
  • [TRTLLMINF-172][infra] Block multi-GPU tests when single-GPU tests fail by @ZhanruiSunCh in #15971
  • [TRTLLMINF-113][infra] Refine Setup/Initialize timeout granularity by @ZhanruiSunCh in #15953
  • [https://nvbugs/6384633][fix] Fix NCCL Shutdown by @belgarten-nv in #15892
  • [TRTLLM-13409][feat] proxy fast-death detection + sticky EngineDeadError by @JunyiXu-nv in #15816
  • [None][test] bench_moe: add nsys capture-range support (--nsys) by @leslie-fang25 in #16113
  • [None][test] KV cache manager v2: add V2 + VSWA multi-GPU test coverage by @eopXD in #16114
  • [None][fix] Improve scope conflict handling in Selector class by @yufeiwu-nv in #16181
  • [https://nvbugs/6221055][fix] added four GB300 rows to agg_unit_mem_df.csv by @tensorrt-cicd in #14785
  • [https://nvbugs/6379636][fix] Fix Gemma4 MoE weight loading by @2ez4bz in #16108
  • [https://nvbugs/6411873][fix] DSV4: size KV constraint with num_extra_kv_tokens by @xguannv in #16173
  • [https://nvbugs/5970614][fix] Sync CTA before PDL trigger in quantize_with_block_size by @tianyuxbear in #14668
  • [https://nvbugs/6208457][fix] Harden spawn proxy HMAC key env variable handling by @yibinl-nvidia in #16035
  • [TRTLLMINF-191][infra] Make S3 stdout echo opt-in by @niukuo in #16182
  • [TRTLLM-14040][fix] VisualGen shared-tensor handle: same-process (external-launch) handoff by @luyiyun1021 in #16050
  • [None][refactor] BREAKING: rename server args by @2ez4bz in #16091
  • [https://nvbugs/6342844][fix] Prevent disaggregated KV transfer stalls by @chienchunhung in #15737
  • [None][chore] Remove pre-installed opencv by @tburt-nv in #16206
  • [https://nvbugs/6384375][fix] Cap MXFP4 Hopper swizzle transient GPU memory during MoE weight load by @dongfengy in #16125
  • [TRTLLM-13035][feat] Native /v1/embeddings dynamic batching for encoder-only models by @xwang233 in #15424
  • [https://nvbugs/6427240][fix] Reserve MTP draft tokens in scheduler for one-model speculative decoding by @jhaotingc in #16101
  • [TRTLLM-14138][perf] Add fused kernels for Gemma4 serving by @kaiyux in #16074
  • [None][fix] trtllm-serve VisualGen: bind HTTP port only on rank 0 in multi-rank launch by @luyiyun1021 in #16049
  • [TRTLLM-13776][feat] Add Piecewise CUDA Graph support for Qwen3.5/Qwen3.6 MoE model by @nv-guomingz in #15900
  • [TRTLLM-14198][refactor] Make FlashInfer a hard dependency for the Torch sampler by @zhaoyangwang-nvidia in #16160
  • [None][fix] AutoDeploy: return fused_weight_dims so fused QKV split sizes are rescaled under TP by @CodersAcademy006 in #15351
  • [None][fix] Fix indexer-k-cache transfer for disagg by @Tabrizian in #16197
  • [None][fix] Correct RocketKV KT cache byte accounting for FP8 KV cache by @eopXD in #16225
  • [None][perf] serve: opt-in msgspec msgpack transport for disagg orchestrator->worker request body by @Tabrizian in #16126
  • [None][feat] Add an opt-in raw-weight cache to the HF weight loader by @sunnyqgg in #16054
  • [https://nvbugs/6430674][fix] Surgical revert of PR #15838's dispatch guard removal: re-add… by @chenfeiz0326 in #16167
  • [None][fix] source DSA metadata from sparse params by @liji-nv in #16236
  • [None][infra] Waive 7 failed cases for main in post-merge 2835 by @trtllm-agent in #16292
  • [https://nvbugs/6323074][fix] fix flaky hang issue for disagg gen-onl… by @bo-nv in #16172
  • [None][test] Waive 9 failed cases for main in QA CI by @trtllm-agent in #16271
  • [None][test] Waive 6 failed cases for main in QA CI by @trtllm-agent in #16295
  • [None][test] Waive 4 failed cases for main in QA CI by @trtllm-agent in #16296
  • [None][test] Waive 1 failed cases for main in QA CI by @trtllm-agent in #16283
  • [https://nvbugs/6438658][fix] Avoid false disagg benchmark KV exhaustion by @chienchunhung in #16248
  • [https://nvbugs/6330273][fix] Reserve worst-case SWA slots to avoid single-request deadlock by @lowsfer in #15588
  • [None][feat] BREAKING: add per-model KV cache manager v2 auto selection by @yizhang-nv in #15823
  • [TRTLLM-14138][perf] Make FlashInfer decode plans sync-free with host-built page tables by @kaiyux in #16073
  • [None][fix] Fix fused mHC output reuse and extend compressor next_n by @heyuhhh in #16221
  • [https://nvbugs/6428113][fix] Fix TEP token-count handling in MoE Test by @guqiqi in #16293
  • [None][feat] Support externally provided MPI sessions with explicit ownership by @sunnyqgg in #16053
  • [None][perf] offload chat template rendering into async by @yechank-nvidia in #15284
  • [None][feat] add commit min snapshot for SSM reuse by @lowsfer in #15752
  • [None][chore] Clean deprecated CppMambaCacheManager by @bo-nv in #15533
  • [https://nvbugs/6437410][chore] Waive a failed test in pre-merge CI by @shuyixiong in #16308
  • [https://nvbugs/6379333] [chore] Unwaive test cases by @kaiyux in #16180
  • [None][test] Restrict gen-worker per-iter mean to steady-state iterations by @chenfeiz0326 in #16298
  • [https://nvbugs/6367792][fix] unwaive Nano AutoDeploy tests by @tcherckez-nvidia in #16285
  • [None][chore] Fix lock_infra_error by @yufeiwu-nv in #15213
  • [None][fix] Fix VSWA gate in KVCacheManager window-size resolution by @eopXD in #16233
  • [None][fix] DSA indexer: persistent topk-output buffer to avoid CUDA-graph stale-pointer IMA by @Tabrizian in #16290
  • [None][ci] Disable B300 stages during aws-pdx cluster maintenance by @dpitman-nvda in #16334
  • [None][chore] Remove unused token_range and is_last_slice parameters from _create_kv_slice by @Tabrizian in #16083
  • [TRTLLM-11558][feat] BREAKING: Acceptance rate based speculation off in one model path by @zheyuf in #12905
  • [None][perf] Cute DSL GVR Top-K: short-row remove cluster sync in run_one_row by @limin2021 in #15835
  • [None][test] Optimize LLM test startup overhead by @sunnyqgg in #15777
  • [https://nvbugs/6430702][perf] Restore NVLink one-sided A2A fast path by @chienchunhung in #16200
  • [TRTLLM-13696][test] Part1: Add CPU only CI stage by @tongyuantongyu in #15815
  • [https://nvbugs/6115271][fix] Fix LTX-2 Attention Metadata issue by @yibinl-nvidia in #14374
  • [None][chore] Code review group by @juney-nvidia in #16321
  • [https://nvbugs/6422334][fix] In PyExecutor._prepare_draft_requests, gate the py_last_draft_tokens snapshot… by @trtllm-agent in #16121
  • [None][fix] Align dense Qwen3.5-VL SSM cache dtype test with #16065 semantics by @moraxu in #16264
  • [https://nvbugs/6445322][fix] Restore the pre-#16108 baseline - accuracy: 56.667 alongside the existing… by @trtllm-agent in #16310
  • [TRTLLM-14040][fix] Fix multi-GPU VisualGen response IPC: REBUILD_LOCAL handle crossed a process boundary by @luyiyun1021 in #16326
  • [None][infra] Waive 3 failed cases for main in post-merge 2836 by @trtllm-agent in #16356
  • [None][perf] Memoize multimodal run metadata and drop identity block-offset gather by @eopXD in #16232
  • [https://nvbugs/6434572][fix] Add a new pytorch_model_config.py pattern block covering the six 4-GPU… by @trtllm-agent in #16199
  • [None][test] Skip test_disaggregated_llama_context_capacity fp8 on hopper by @jieli-matrix in #16357
  • [TRTLLM-14234][perf] LTX-2: replicate audio stream across Ulysses ranks by default by @luyiyun1021 in #16228
  • [TRTLLM-14054][perf] Load Qwen3-Next GDN in_proj in dense layout and add a multi-row gated RMSNorm by @kaiyux in #16314
  • [https://nvbugs/6438658][fix] Fix MLA KV cache estimation sizing by @jiaganc in #16311
  • [None][test] Waive 1 failed cases for main in QA CI by @trtllm-agent in #16243
  • [None][infra] Waive 1 failed cases for main in pre-merge 47605 by @trtllm-agent in #16366
  • [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend by @Wanli-Jiang in #15918
  • [https://nvbugs/6127669][fix] Fix layer-wise benchmarks performance alignment test by @kaiyux in #16212
  • [None][feat] Enable agent serving evaluation via trace-replay on Scaffolding by @KleinBlueC in #14397
  • [None][feat] Integrate KVCMv2 commit_min_snapshot in runtime by @jiaganc in #16185
  • [TRTLLM-14054][fix] LMHead: pass true full dims to Linear for quantized TP>1 by @kaiyux in #16352
  • [None][test] Waive 8 failed cases for main in QA CI by @trtllm-agent in https://github.com/NVIDIA/TensorRT-LLM/pull/16061
  • [None][chore] Remove deprecated rules by @juney-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/16373
  • [TRTLLM-14054][perf] Qwen3.5-VL: pass the inner LM's normalized model_config to the weight mapper by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/16353
  • [None][fix] Seq-slot pool overlap headroom with consistent slot-indexed buffer sizing by @qiaoxj07 in https://github.com/NVIDIA/TensorRT-LLM/pull/16279

New Contributors

Full Changelog: v1.3.0rc20...v1.3.0rc21

Contributors

Superjomn, reasonsolo, and 115 other contributors
Assets 2
Loading