[Roadmap] vLLM Roadmap Q3 2026 #48168

In #32455, we broke down vLLM's goal into various special interest groups (SIGs). Please find below each SIG's Q3 roadmap. You can find regular meetings of these SIGs on the public vLLM calendar.

Key takeaway

• Agentic Workload: In Q2, vLLM doubled down on core performance reaching TensorRT-level speed-of-light performance for top models, along with production features such as disaggregated serving and KV offloading maturity. In Q3, we focus on production agentic workload and high-interactivity premium tokens.
• Ecosystem: The ecosystem has been stronger than ever: quantization (ModelOpt and llm-compressor), speculations (Speculators and TorchSpec), and RL (vime, Prime-RL, Nemo-RL, etc.) are three established main ecosystems surrounding core vLLM. vLLM-Omni has reached maturity with a flywheel of model + hardware + optimization support.
• OSS Maintenance: The rise of coding agents brought new challenges to open source maintenance. We are re-engineering vLLM core abstractions (Flat Model), developer infrastructure (nightly model quality test), security processes (CVE and disclosure), and model / hardware / features lifecycle management: when to add, who to contact, and when to remove.

SIG Core (Meeting Notes)

• Complete Flat Model Migration for the top 20 model architectures and ensure new day-0 models are Flat Model only.
• Complete Model Runner V2 migration and ensure new day-0 models are MRV2 only, deprecate MRV1.
• Improve production stability and failure ergonomics.
• Scheduler refactoring.
• KV Cache Manager redesign.
• Mark Rust Frontend and refactored tool-calling implementation as production ready.
• Reduce cold start time [Roadmap] Weight-loading-bound startup #48193

SIG Large Scale Serving (Meeting Notes)

• SOTA performance on AgentX for top model disaggregated serving with KV offload and tuned PD recipes.
• Production-ready distributed and multi-tier KV cache offloading.

• Long context:
  • Efficiently support various context parallelisms for SOTA models.
• Agent-oriented Prefix Caching Policies:
  • Agent hints, such as Session-ID and Correlation-ID, for vLLM to understand multi-turn agents and their subagents.
  • Targeted customization for scheduling and KV cache management, such as prefix cache prefetching, eviction, and selective offloading.
• Routing/scheduling:
  • Support KV Events in peer-to-peer distributed KV caches like Mooncake.
  • KV Events support for tiered offloading.

• PD: tuned recipes for SOTA models with KV cache offloading.
• Improvements to Elastic EP.
• Faster scale-up and scale-down via asynchronous scaling.
• Fault-tolerance EP recovery via restart and scale-down.
• AMD Parity: RCCL/RDMA path for Disagg + KV offload + Elastic EP + Mooncake P2P/tiered KV events @ChuanLi1101
• AMD Sprint: DeepSeek Long Context CP on MI3xx @ChuanLi1101

SIG Model Performance (Meeting Notes)

• Infrastructure: enhance coverage for release-gating accuracy and performance regression tests, automatic trace upload and analysis, and expanded hardware coverage and workloads tracked.
• Transition from Q2's forum style to a sprint-based mechanism for each model, listing out concrete tasks. The current sprints are GLM5.2 and DSpark. Help wanted.

SIG Spec Decode (Meeting Notes)

• Long-context tuning: efficiently support sliding-window, sparse, and hybrid/linear drafting modes with quantized KV caches.
• Target higher acceptance length (>5) in common workloads, pushing 1000 TPS as a common target with the --performance-mode inferactivity option.
• Support SOTA draft architectures: implement and optimize DFlash, DFlare, and DSpark with feature compatibility and tuned kernels.
• User Experience: improve UX for optimized speculative decoding configs, possibly enabling speculation by default. Improve robustness with tool calling + reasoning and structured outputs.
• Dynamic Speculative Decoding: improve speculative efficiency by reducing verification overhead, especially for high-concurrency and heterogeneous serving workloads.
• AMD: DSpark support and Quark ecosystem.

SIG CI (Meeting Notes)

• Bring time to signal down to 30 minutes.
• Automatic quarantine for flaky tests (per hardware backend)
• Multi-node CI and performance benchmark.
• Refactor alerts/notifications system.

• Infra:
  • Migrate away from AWS L4s.
  • Slurm setup to power multi-node testing.
  • Monitor / health check for compute and take unhealthy nodes off before failures.
  • MI3xx as a CI presence and path to gating.

SIG Quantization

• KV-cache compression for agentic and long-context workloads: move quantized KV cache from "feature availability" to production-grade serving. Strengthen FP8, NVFP4, INT2/4, TurboQuant/HIGGS-style, and emerging rotation-based KV compression paths across hybrid attention layouts, disaggregated deployments, and tiered KV offload.
• Unified quantization dispatch and inspectability: complete the QuantKey / dynamic QuantConfig refactor so backend selection is deterministic, debuggable, and model-aware.
• Manual quant fusion and Flat Model readiness: finish the migration from compiler-driven quant fusions to explicit model-code fusions using QuantizedActivation, including RMSNorm+quant, activation+quant, all-reduce+residual+RMSNorm+quant, RoPE+KV-cache-write+quant, and MLA attention quant fusions.

vLLM-Omni

• Productionization of real-time full-duplex models, such as JoyVL and MiniCPM-o.
• Optimization of top-priority interaction and Omni models, such as Cosmos3 and Qwen3-Omni.
• Video generation: streaming video generation and FastVideo integration.
• ModelConfig refactoring and codebase restructuring.

RL Ecosystem

• [Roadmap] 2026 Q3 vLLM × RL #48314

Please leave feedback and comments, and work directly with the relevant SIG areas for deeper collaboration.