[Roadmap] vLLM Roadmap Q3 2026

[simon-mo]

In #32455, we broke down vLLM's goal into various special interest groups (SIGs). Please find below each SIG's Q3 roadmap. You can find regular meetings of these SIGs on the public vLLM calendar.

Key takeaway

• Agentic Workload: In Q2, vLLM doubled down on core performance reaching TensorRT-level speed-of-light performance for top models, along with production features such as disaggregated serving and KV offloading maturity. In Q3, we focus on production agentic workload and high-interactivity premium tokens.
• Ecosystem: The ecosystem has been stronger than ever: quantization (ModelOpt and llm-compressor), speculations (Speculators and TorchSpec), and RL (vime, Prime-RL, Nemo-RL, etc.) are three established main ecosystems surrounding core vLLM. vLLM-Omni has reached maturity with a flywheel of model + hardware + optimization support.
• OSS Maintenance: The rise of coding agents brought new challenges to open source maintenance. We are re-engineering vLLM core abstractions (Flat Model), developer infrastructure (nightly model quality test), security processes (CVE and disclosure), and model / hardware / features lifecycle management: when to add, who to contact, and when to remove.

SIG Core (Meeting Notes)

• Complete Flat Model Migration for the top 20 model architectures and ensure new day-0 models are Flat Model only.
• Complete Model Runner V2 migration and ensure new day-0 models are MRV2 only, deprecate MRV1.
• Improve production stability and failure ergonomics.
• Scheduler refactoring.
• KV Cache Manager redesign.
• Mark Rust Frontend and refactored tool-calling implementation as production ready.
• Reduce cold start time [Roadmap] Cold Start Q3 2026 #48193

SIG Large Scale Serving (Meeting Notes)

• SOTA performance on AgentX for top model disaggregated serving with KV offload and tuned PD recipes.
• Production-ready distributed and multi-tier KV cache offloading.

• Long context:
  • Efficiently support various context parallelisms for SOTA models.
• Agent-oriented Prefix Caching Policies:
  • Agent hints, such as Session-ID and Correlation-ID, for vLLM to understand multi-turn agents and their subagents.
  • Targeted customization for scheduling and KV cache management, such as prefix cache prefetching, eviction, and selective offloading.
• Routing/scheduling:
  • Support KV Events in peer-to-peer distributed KV caches like Mooncake.
  • KV Events support for tiered offloading.

• PD: tuned recipes for SOTA models with KV cache offloading.
• Improvements to Elastic EP.
• Faster scale-up and scale-down via asynchronous scaling.
• Fault-tolerance EP recovery via restart and scale-down.
• AMD Parity: RCCL/RDMA path for Disagg + KV offload + Elastic EP + Mooncake P2P/tiered KV events @ChuanLi1101
• AMD Sprint: DeepSeek Long Context CP on MI3xx @ChuanLi1101

SIG Model Performance (Meeting Notes)

• Infrastructure: enhance coverage for release-gating accuracy and performance regression tests, automatic trace upload and analysis, and expanded hardware coverage and workloads tracked.
• Transition from Q2's forum style to a sprint-based mechanism for each model, listing out concrete tasks. The current sprints are GLM5.2 and DSpark. Help wanted.

SIG Spec Decode (Meeting Notes)

• Long-context tuning: efficiently support sliding-window, sparse, and hybrid/linear drafting modes with quantized KV caches.
• Target higher acceptance length (>5) in common workloads, pushing 1000 TPS as a common target with the --performance-mode inferactivity option.
• Support SOTA draft architectures: implement and optimize DFlash, DFlare, and DSpark with feature compatibility and tuned kernels.
• User Experience: improve UX for optimized speculative decoding configs, possibly enabling speculation by default. Improve robustness with tool calling + reasoning and structured outputs.
• Dynamic Speculative Decoding: improve speculative efficiency by reducing verification overhead, especially for high-concurrency and heterogeneous serving workloads.
• AMD: DSpark support and Quark ecosystem.

SIG CI (Meeting Notes)

• Bring time to signal down to 30 minutes.
• Automatic quarantine for flaky tests (per hardware backend)
• Multi-node CI and performance benchmark.
• Refactor alerts/notifications system.

• Infra:
  • Migrate away from AWS L4s.
  • Slurm setup to power multi-node testing.
  • Monitor / health check for compute and take unhealthy nodes off before failures.
  • MI3xx as a CI presence and path to gating.

SIG Quantization

• KV-cache compression for agentic and long-context workloads: move quantized KV cache from "feature availability" to production-grade serving. Strengthen FP8, NVFP4, INT2/4, TurboQuant/HIGGS-style, and emerging rotation-based KV compression paths across hybrid attention layouts, disaggregated deployments, and tiered KV offload.
• Unified quantization dispatch and inspectability: complete the QuantKey / dynamic QuantConfig refactor so backend selection is deterministic, debuggable, and model-aware.
• Manual quant fusion and Flat Model readiness: finish the migration from compiler-driven quant fusions to explicit model-code fusions using QuantizedActivation, including RMSNorm+quant, activation+quant, all-reduce+residual+RMSNorm+quant, RoPE+KV-cache-write+quant, and MLA attention quant fusions.

vLLM-Omni

• Productionization of real-time full-duplex models, such as JoyVL and MiniCPM-o.
• Optimization of top-priority interaction and Omni models, such as Cosmos3 and Qwen3-Omni.
• Video generation: streaming video generation and FastVideo integration.
• ModelConfig refactoring and codebase restructuring.

RL Ecosystem

• [Roadmap] 2026 Q3 vLLM × RL #48314

---

Please leave feedback and comments, and work directly with the relevant SIG areas for deeper collaboration.

[github-actions]

CC @hongxiayang @tjtanaa @vllmellm for ROCm-related issue

[LCAIZJ]

Hi @simon-mo, KV Offloading was mentioned multiple times in the vLLM Q3 Roadmap. I'm wondering if there's a detailed design proposal available for this feature? We've also previously submitted a proposal related to KV Offloading #48203, and I'd like to check whether it aligns with the current roadmap.

[vrdn-23]

OSS Maintenance: The rise of coding agents brought new challenges to open source maintenance. We are re-engineering vLLM core abstractions (Flat Model), developer infrastructure (nightly model quality test), security processes (CVE and disclosure), and model / hardware / features lifecycle management: when to add, who to contact, and when to remove.

I'd be interested in understanding what are some new approaches that the vLLM core team is doing to make sure PRs are not abandoned without review. I realize that there are a lot of noise that has come with the advent of coding agents (and that the cost of that comes with some stuff just escaping us), but I am hoping that now seems like a good time for the team to come and consolidate on an approach that can hopefully make use of community resources to identify whether a PR is worth the maintainer's time in terms of 1) whether it actually solves a problem vLLM has 2) whether it is a legitimate PR/improvement/feature and not just slop that is not welcome.

It can be something simple as using emojis to upvote or downvote PRs and have two PRs with most upvotes reviewed (and maybe? merged) every week, or have some concept of community "verifiers" who have done contributions in the past, and have the bare minimum skills to verify vLLM PRs by running them and verifying that the PR fixes a certain issue or correctly identifies a bug and solves it.

I do think that we need some sort of a more organized mechanism for getting attention from maintainers on community PRs (that may not follow the current roadmap but are worthwhile by nature) that doesn't involve luck or how free someone is to leave a review.

cc @simon-mo @DarkLight1337 @robertgshaw2-redhat @russellb