how to set max frames and fps at the same time.

[WudiJoey]

Input is a fps=30, length=61 seconds , 720x1280 video. VLLM online inference with extra_body

mm_processor_kwargs = { 
      "fps": 1,
      "do_sample_frames": True,
 }

Prompt is "描述视频内容”.
Finallay, vllm return {'prompt_tokens': 11981, 'completion_tokens': 711, 'total_tokens': 12692}) (how the token num is calculated ???)

When I add max_frames=16 in mm_processor_kwargs, expecting using fps=1 but total frame num not exeed 16, vllm return errors: {'error': {'message': 'The timestamps length(30) should be equal video length (8).', 'type': 'Internal Server Error', 'param': None, 'code': 500}}

I am so confused how vllm sampling video inside. https://github.com/huggingface/transformers/blob/9aab965b1e61d92d402809bd467c317ec464e560/src/transformers/models/qwen3_vl/video_processing_qwen3_vl.py#L99

Maybe canceling video base64 jpgs is a bad idea...

[WudiJoey]

I test on fps=2, 'prompt_tokens': 12145, compared to fps=1, 'prompt_tokens': 11981, what is the difference?

[JJJYmmm]

• (how the token num is calculated ???), token num \approx num_frames*(num_timestamp + num_visual_tokens_perframe)
• The timestamps length(30) should be equal video length (8)., currently vllm only supports fps.
• I test on fps=2, 'prompt_tokens': 12145, compared to fps=1, 'prompt_tokens': 11981, what is the difference?, for different fps settings, the frames num varies, but the resolution for each frame is different (that's determined by smart_resize in hf processor).

So we recommend using qwen-vl-utils do sample the videos.

[WudiJoey]

NO, qwen-vl-utils cannot work with vllm online. I read it from another issue.

qwen-vl-utils can control fps、max_frames、max_pixels(for each frame) at the same time. But the processing and parameters inside VLLM are different from it.

Qwenvl 3 with online vllm, only show example using fps and based on my test, max_frames is not work and max_pixels is for whole video.

Finally, I must obtain the fps and duration of the video to calculate the most suitable fps and accurately control the resolution.
So sad~ Qwen vl 2.5 do the better work with online vllm.

[JJJYmmm]

Yes, I mean using qwen-vl-utils with offline inference.
Indeed, online serving of qwen3vl is more complex than that of qwen2.5vl, primarily due to the introduction of timestamps, along with additional complexities from vLLM features such as multimodal caching. We will continue to optimize this in future updates.

[WudiJoey]

Got it，thanks and wish for better use for video one day！

[YounesElhjouji]

@JJJYmmm Any updates on support for more control on video sampling? If the issue is just with parameters taking effect, maybe I can contribute a fix to the repo. If the issue is deeper and relates to the model architecture, please tell me a little about the root cause so I can update my usage or maybe use the vl models outside vllm or through sglang.