[RFC]: vLLM-Omni Diffusion Module — Q2 2026 Roadmap

[wtomin]

Motivation.

This roadmap consolidates open RFC items, and community priorities for the vllm-omni diffusion model module. Items overlapping with the Q2 Roadmap RFC #2136 are treated as highest urgency.

P0 must-have · P1 target Q2 · P2 stretch/Q3 · — open slot · (RFC needed) = no tracking issue yet, open one to claim

Want to contribute?

• See [RFC]: Continuous Recipe Support for Diffusion Models #3629 for open diffusion model recipe tasks.
• See [RFC]: Continuous Diffusion Model Acceleration Support #1217 for open diffusion feature tasks.
• See [RFC]: Continuous Quantization Support  #1854 for open diffusion quantization tasks.
• See [RFC]: L4 e2e tests of diffusion models and diffusion features (continuous maintanance) #1832 for diffusion testing tasks.

---

Proposed Change.

---

1. Diffusion User Interface & Guidance

Goal: Reduce the barrier to entry for diffusion models through recipes, guardrails, and auto-generated capability documentation.

• [P0] Diffusion Model Recipes — Publish end-to-end usage recipes covering recommended configurations, and performance tuning for popular diffusion models. RFCs: [RFC]: Wan2.2 Performance Optimization Roadmap on vLLM-Omni  #1355, [RFC]: FLUX.2-dev Model Acceleration Support #1806, [RFC]: HunyuanImage Model deployment optimization #2015; PRs:[Recipe] Add Fish Speech S2 Pro 2-GPU deploy profile #3323, [Recipes]update Wan2.2-I2V gpu part #3271, [Docs] Add recipe for Helios #3114
• [P0] Pipeline Parameter Validation — Add pre-flight checks that detect invalid parameter combinations at startup, e.g. incompatible parallelism modes or incorrect diffusion sampling params. RFCs: [RFC]: Add Diffusion Pipeline Protocol / Base Class #2189,[RFC]: [Config Refactor][2/2] Pipeline + Deploy Config System #2072, PRs: Add Diffusion Pipeline Base Class #2653
• [P0] Pipeline × Feature Support Registry — Build a structured compatibility registry that maps each pipeline to its supported acceleration features. ([RFC]: Add Diffusion Pipeline Protocol / Base Class #2189)
• [P0] [RFC]: Fix + Standardize DiT Config/Init Patterns #1456 — Relevant high-priority community issue; scope follows the latest issue description.

---

2. Systematic Core Features

2.1 DiT Continuous Batching & Step-Level Scheduling

Goal: Enable fine-grained scheduling of diffusion denoising steps to improve throughput and support streaming job management.

• [P0] Abort Request — Support cancellation of in-flight diffusion inference requests. ([Feat] Support step-boundary abort in diffusion #1769)
• [P0] Step-wise & Continuous Batching — Decompose the denoising loop into step-level scheduling units, allowing continuous batching for diffusion models. RFCs: [RFC]: Refactor engine/runner/pipeline to support step-wise and continuos batching  #874, [Diffusion] Refactor CFG parallel for extensibility and performance #2063, PRs: [Draft] diffusion continuous batching #2652, [Feat] DiffusionEngine Support async batch infer  #2729
• [P1] Data Parallelism for Diffusion — Add data-parallel support so that multiple requests can be dispatched to independent DP replicas concurrently. ([Feature]: Support data parallel size for diffusion model #1706)

2.2 Reinforcement Learning Support

Goal: Enable diffusion models to serve as generative policies within RL rollout.

• [P0] RL Inference Interface — Expose inference primitives required by RL training, including log-probability computation and gradient passthrough. ([RFC]: Reinforcement learning support for multi-stage models (Qwen3-Omni) in vLLM-Omni #2357, [RFC]: Reinforcement learning support on vllm-omni #778)
• [P1] LoRA Enhancements — LoRA loading acceleration, multi-LoRA switching, and non-PEFT format support. RFCs: [Performance]: Redundant LoRA file I/O in multi-GPU diffusion inference #2198, [RFC]: LyCORIS Adapter Support for Diffusion Models (LoKr, LoHa, and beyond) #2150, [RFC]: Multi-LoRA Composition for Diffusion Models #2149, [Feature]: Allow Non-Peft Format for LoRA #1396, [Feature] Support multi-LoRA composition for diffusion models #2309, PRs:[Feature]: Adds distilled LoRA support for diffusion models #2783
• [P1] Diffusers Backend — Instead of vLLM-Omni, using diffusers as the inference backend. RFCs: [RFC]: Fix + Standardize DiT Config/Init Patterns #1456, PRs: [feat]: General diffusers adapter backend to run diffusion models #2724

2.3 Multi-Stage Orchestration

Goal: Allocate GPU/CPU resources across pipeline stages (text encoder → DiT → VAE) to accelerate diffusion models.

• [P1] Cross-Stage Orchestration & Resource Allocation — Optimized stage deployment for models like Bagel that benefit from cross-stage resource splitting. RFCs: [Feature]: Support VAE as a Separate Stage to Reduce GPU Memory Pressure in Diffusion Pipelines #2089 PRs: feat(diffusion): support Qwen-Image disaggregated VAE #3208

---

3. World Model Support

Tracking: #1987

Goal: Provide the infrastructure needed for world models (autoregressive diffusion, VLM, 3D, etc.) to support various features.

• [P0] World Model Support — Support the inference acceleration of World Models, such as interactive video generation model, VLA(Vision-Language-Action) model. For example, DreamZero RFCs: [RFC]: World Model Support #1987
• [P1] PagedAttention & KV Cache for Autoregressive Diffusion — Wire KV cache reuse into AR-diffusion inference to avoid full recomputation over growing context windows. RFCs: [RFC]: PagedAttention and KV Cache for Autoregressive Diffusion #2305, PRs: [Feature] HunyuanImage-3.0 AR->DiT KV-cache reuse for image editing (IT2I) #2949
• [P1] Semi-Causal Attention Support — Support flexible switching between autoregressive and full attention; requires attention backend changes. PRs: [Feature] [HunyuanImage3] Use FlashAttention for image denoising #1975 , [Feature] hunyuanimage support flash attn #2981
• [P1] Pipeline Parallelism & Stream Batch — Introduce pipeline parallelism and stream batching for real-time video generation, like StreamDiffusionV2. RFCs: [RFC]: Pipeline Parallelism & Stream Batch for Real-Time Video Generation #2280 PRs: [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 #2322, [Feature] Temporal Pipeline Parallelism & Stream Batch for Real-Time Video #3099
• [P1] Streaming Inputs & Outputs — Frame-level streaming generation, instantaneous condition insertion, and checkpoint/resume for long-horizon inference. RFCs: [RFC] Streaming Video Input for Omni-Modal Real-Time Interaction #2201, PRs:[Feature] Streaming video input with EVS frame filtering (RFC #2201 Phase 2-4) #2342, Realtime video input/output #3157

---

4. Model Coverage Expansion

Goal: Merge all in-progress model PRs and systematically fill feature-coverage gaps in the existing model matrix.

• [P0] In-Progress Model PRs — Complete and merge six actively developed model-support PRs. Full test coverage required before merge. ([Model] Add Index-AniSora I2V support (V1 5B + V2 14B) #877, [Model] Add Dynin-omni model in vllm-omni #1759, [Model] : support FLUX.2-klein-9b-kv #1866, [RFC]: HunyuanImage Model deployment optimization #2015, [Model] Add PrismAudio pipeline skeleton with request validation.  #2181, [RFC]: FLUX.2-dev Model Acceleration Support #1806)
• [P0] Quantization Coverage Expansion — Extend FP8/Int8 quantization support to additional image and video generation models. ([RFC]: Continuous Quantization Support  #1854)
• [P1] Feature Matrix Completion — Systematically extend acceleration features to models with incomplete coverage; run feature compatibility tests. ([RFC]: Continuous Diffusion Model Acceleration Support #1217, [WIP] Feature Compatibility Offline Test #2083)

---

5. Online Serving & Entrypoint

Goal: Complete missing API endpoints, harden existing ones, and improve production reliability for long-running generation jobs.

• [P0] Text-to-Audio Endpoint — Deliver a /v1/audio/generations endpoint for audio diffusion models (e.g., Stable-Audio-Open) aligned with the OpenAI Audio API. (Fix "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" #1794)

---

6. Testing, CI/CD & Documentation

Goal: Establish continuous end-to-end quality assurance and keep documentation in sync with the codebase.

• [P0] Continuous L4 E2E Functional Testing — Stand up CI tests for diffusion models (potentially using smaller random-weight models). ([RFC]: L4 e2e tests of diffusion models and diffusion features (continuous maintanance) #1832)
• [P0] Performance & Accuracy Benchmark Testing — Performance regression and accuracy monitoring tests in CI. ([CI] hunyuanimage benchmark tests for performance regression tracking #2185) (RFC needed)
• [P1] L5 Testing — Test the stability and reliabilty for long-term severing (RFC needed)
• [P1] Docs Refinement — Overhaul offline inference and online serving documentation with complete, runnable examples across multiple settings. ([RFC]: Diffusion Offline Examples Docs & Test #1244)

---

CC List.

cc @hsliuustc0106 @Yikun @wangxiyuan @Fishermanykx @jiangmengyu18 @blian6 @SamitHuang @ZJY0516 @linyueqian @david6666666 @Gaohan123 @tzhouam @princepride @fhfuih

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[princepride]

I want to add some additional information about the RL support. I think we need add RL support for two typical multi-stage models (Bagel and Qwen3-Omni). For Bagel, its DiT must support a custom scheduler and be able to output logprobs. It seems unnecessary to output hidden_states if we are not using PPO. For Qwen3-Omni, the talker and code2wav components also need to support outputting hidden_states and logits.

Regarding weight synchronization, I'm not sure if we currently support partial weight synchronization. Since the weights of certain stages remain completely frozen during training, our stage-level strategy could theoretically skip the weight updates for those specific stages. Furthermore, given that weight offloading can be done at a fine-grained level for specific modules, I'm wondering if we can also perform fine-grained weight updates that only target specific weights.

[princepride]

We also need support VLA model, we already have an issue about our first VLA model support: #1948

[hsliuustc0106]

for the ci/cd, i think we need add L5 test, e.g., long run test for P0 models

[wtomin]

For Bagel, its DiT must support a custom scheduler and be able to output logprobs. It seems unnecessary to output hidden_states if we are not using PPO. For Qwen3-Omni, the talker and code2wav components also need to support outputting hidden_states and logits.

I think it is relevant to custom pipelines. Can you check it already satisfies the needs? @princepride

I think the partial weight synchronization is relevant to vllm_omni/diffusion/model_loader/diffusers_loader.py. I will check further.

[princepride]

For Bagel, its DiT must support a custom scheduler and be able to output logprobs. It seems unnecessary to output hidden_states if we are not using PPO. For Qwen3-Omni, the talker and code2wav components also need to support outputting hidden_states and logits.

I think it is relevant to custom pipelines. Can you check it already satisfies the needs? @princepride

I think the partial weight synchronization is relevant to vllm_omni/diffusion/model_loader/diffusers_loader.py. I will check further.

Thanks, I already noticed this feature.

[TKONIY]

Do we need to add world model support into Q2?

• [RFC]: World Model Support #1987