server : reuse image features within a SystemOne request

[mshr-h]

When a /v1/systemone request contains several questions about the same image, the server can repeat vision encoding and projection for each question. This adds redundant work as the number of questions grows.

I propose computing the image features once and sharing them across questions within the request. The request would own these features and release them when processing finishes.

I tested a prototype covering both causal and non-causal paths with:

• d1-omni-600M: FP16 and Q8_0
• vllm-d3-edge, vllm-d3-nano, and d1-3B: Q8_0

Measurements used Jetson AGX Orin, CUDA Release, parallel=1, and context/batch/ubatch sizes of 4096. Each condition used the same 30 images at 448x448 over three rounds, excluding warmup and disabling previous-request caching.

For requests with 2, 4, and 8 questions, encode calls decreased from 2, 4, and 8 to 1. Across the 1,350 Before/After request pairs, answers and probabilities matched exactly.

For example, d1-omni-600M FP16 with four questions reduced median server time, including CUDA synchronization, from 246.5 ms to 159.3 ms (-35.37%). Single-question requests retained one encode call; D3-edge Q8_0 showed a 1.16% increase in median server time. Other models, hardware, and parallel>1 remain untested.

Related work:

• mtmd: Expose helper_decode_image_chunk #13366 : a helper for consuming precomputed image embeddings.
• Eval bug: Image prompt cache never hits for hybrid SSM models (Qwen3.6) when only the text suffix changes #26994 : repeated image processing when the text suffix changes across requests.
• server: allow checkpoint after last media #30201 : checkpoints after media to address that issue.

AI assistance: Codex generated the implementation, tests, and benchmark scripts. All changes were reviewed by a human.