vllm 推理 TimeLens-8B报错

[LiYufengzz]

(APIServer pid=454) INFO: 127.0.0.1:35558 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] Error in preprocessing prompt inputs
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] Traceback (most recent call last):
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/context.py", line 267, in call_hf_processor
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/transformers/models/qwen3_vl/processing_qwen3_vl.py", line 170, in call
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] videos_inputs = self.video_processor(videos=videos, **output_kwargs["videos_kwargs"])
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/transformers/video_processing_utils.py", line 206, in call
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] return self.preprocess(videos, **kwargs)
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/transformers/video_processing_utils.py", line 387, in preprocess
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] preprocessed_videos = self._preprocess(videos=videos, **kwargs)
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/transformers/models/qwen3_vl/video_processing_qwen3_vl.py", line 242, in _preprocess
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] patches = patches.view(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] RuntimeError: shape '[1, 35, 2, 3, 45, 2, 16, 33, 2, 16]' is invalid for input of size 326592000
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311]
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] The above exception was the direct cause of the following exception:
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311]
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] Traceback (most recent call last):
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 295, in render_chat_request
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] conversation, engine_prompts = await self._preprocess_chat(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 982, in _preprocess_chat
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] (conversation,), (engine_prompt,) = await renderer.render_chat_async(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/renderers/base.py", line 764, in render_chat_async
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] self.process_for_engine(prompt, arrival_time) for prompt in tok_prompts
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/renderers/base.py", line 656, in process_for_engine
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] engine_prompt = self._process_singleton(prompt)
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/renderers/base.py", line 632, in _process_singleton
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] return self._process_tokens(prompt) # type: ignore[arg-type]
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/renderers/base.py", line 577, in _process_tokens
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] inputs = self._process_multimodal(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/renderers/base.py", line 563, in _process_multimodal
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] mm_inputs = mm_processor.apply(mm_processor_inputs, mm_timing_ctx)
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1682, in apply
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ) = self._cached_apply_hf_processor(inputs, timing_ctx)
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1471, in _cached_apply_hf_processor
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ) = self._apply_hf_processor_main(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1288, in _apply_hf_processor_main
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] mm_processed_data = self._apply_hf_processor_mm_only(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1246, in _apply_hf_processor_mm_only
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] _, mm_processed_data, _ = self._apply_hf_processor_text_mm(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1173, in _apply_hf_processor_text_mm
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] processed_data = self._call_hf_processor(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_vl.py", line 997, in _call_hf_processor
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] video_outputs = super()._call_hf_processor(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 1130, in _call_hf_processor
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] return self.info.ctx.call_hf_processor(
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] File "/mnt/hwnasbj/huixuan/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/multimodal/processing/context.py", line 296, in call_hf_processor
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] raise ValueError(msg) from exc
(APIServer pid=454) ERROR 04-21 02:49:48 [serving.py:311] ValueError: Failed to apply Qwen3VLProcessor on data={'text': '<|vision_start|><|video_pad|><|vision_end|>', 'videos': [[array([[[[225, 211, 212],

用claude code分析，发现可能是这个的问题

想问下，为什么这里相比原始的Qwen3-VL-8B-Instruct多了这么多参数，这个resize似乎应该是true？

[LiYufengzz]

原因查到了，tokenizer_config.json 里的 "do_resize": false 改为true后可以正常推理

[jonasjunzhang]

Hi @LiYufengzz, thanks a lot for reporting this and for carefully tracing the root cause!

You were right that the issue came from how the processor config was exported. In our training/export pipeline, do_resize=False was used as a runtime setting because the visual inputs had already been resized by qwen_vl_utils, but this override was then unintentionally persisted into the released processor config on Hugging Face. That is what caused the incompatibility with vLLM. (We only tried inference with transformers, and we didn't try vLLM inference.)

We have now fixed this in two places:

1. Hugging Face model files:

   • removed the incorrect exported do_resize=False settings from the released processor configs
   • also cleaned up the published config so it no longer keeps the training-time use_cache=False setting

2. Training code:

   • updated the training/export code so do_resize=False is only used at the actual processor call sites, and is no longer persisted into the saved processor defaults

So both the Hugging Face model config and the training code have now been corrected.

Thanks again for catching this and helping us identify the problem!