[Model][Qwen3VL] Cache positional embedding indices

[lgeiger]

Purpose

rot_pos_emb currently runs on the CPU at the beginning of the model forward pass which means that no GPU computation happens in parallel. This PR caches the index computation to help with this. I also moved the index computation to numpy which gave a small ~25-30% improvement to uncached computations as well.

Test Plan

VLLM_TORCH_PROFILER_DIR=./vllm_profile vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct-FP8 --limit-mm-per-prompt.video 0 --max-model-len 10000

Test Result

Before:

After:

[lgeiger]

Using a static methods to make this work nicely with lru_cache

[gemini-code-assist]

Code Review

This pull request introduces a performance optimization for the Qwen3VL model by caching the computation of positional embedding indices. The rot_pos_emb method, which previously ran on the CPU for every forward pass, is refactored to use a new cached static method rot_pos_ids. This new method leverages functools.lru_cache to store results and avoid redundant computations for the same image dimensions. Additionally, the computation logic has been migrated from PyTorch to NumPy, which, as the author notes, provides a slight performance improvement even for uncached calls. The changes are well-implemented, correct, and effectively address the performance bottleneck. The code is clearer and more efficient. I have no further comments.

[ywang96]

LGTM! Left a very small nit

[DarkLight1337]

Please fix the merge conflicts