Draft: Qwen3-VL: support per-request mm_processor_kwargs

[aswinvisva]

…trol

Summary by CodeRabbit

• New Features

  • Added support for passing custom video processing parameters per request through the OpenAI API via a new mm_processor_kwargs field.

• Improvements

  • Enhanced video metadata handling in video preprocessing to provide better control over frame and frame rate configurations.

Description

Pipe mm_processor_kwargs through to the HF processor for Qwen3-VL so downstream video benchmarking can effectively measure metrics w.r.t video FPS accurately.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

Actionable comments posted: 0

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: afecd4d6-2267-47bb-a512-06168d86c422

📥 Commits

Reviewing files that changed from the base of the PR and between 22c7956 and 9c95f88.

📒 Files selected for processing (3)

• tensorrt_llm/_torch/models/modeling_qwen3vl.py
• tensorrt_llm/serve/openai_protocol.py
• tensorrt_llm/serve/openai_server.py

---

📝 Walkthrough

Walkthrough

This PR adds per-request multimodal processor kwargs to the OpenAI API, enabling clients to customize video frame parameters. The request schema accepts optional mm_processor_kwargs, the server routes this to the model, and video preprocessing extracts metadata while resolving fps/num_frames conflicts.

Changes

Per-Request Multimodal Processor Kwargs

Layer / File(s)	Summary
API Contract and Server Forwarding tensorrt_llm/serve/openai_protocol.py, tensorrt_llm/serve/openai_server.py	ChatCompletionRequest adds an optional mm_processor_kwargs field. OpenAIServer.openai_chat conditionally forwards it into the constructed prompt payload.
Video Preprocessing with Metadata and Kwargs tensorrt_llm/_torch/models/modeling_qwen3vl.py	Qwen3VLInputProcessorBase._preprocess extracts per-video metadata from input video_datas and constructs adjusted processor kwargs that explicitly set fps=None when num_frames is specified without fps, ensuring the HF processor uses the caller-provided configuration.

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (3 warnings)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title is a draft work-in-progress and does not follow the required format. It lacks a ticket reference, type annotation, and uses 'Draft:' prefix which is non-standard.	Follow the template format: [JIRA/issue/None][type] Summary. Replace 'Draft:' with a proper ticket reference and include the change type, e.g., '[TRTLLM-XXXX][feat] Qwen3-VL: support per-request mm_processor_kwargs' or use '[None][feat]' if no ticket exists.
Description check	⚠️ Warning	The description is largely incomplete. It provides a brief summary of the change but fails to complete critical sections like Test Coverage, and most PR checklist items are unchecked without verification.	Complete the Test Coverage section specifying relevant tests, verify and check all applicable PR checklist items, ensure API labels are added if applicable, and confirm that code follows TRT-LLM guidelines.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[2ez4bz]

It looks like the missing plumbing was from the API / serve layer, since the code in tensorrt_llm/_torch/models/modeling_qwen3vl.py already forwarded (null) mm_processor_kwargs here. Should we add tests for the code you added in here?