[Doc]: Does llava onevision support VLM multi images?

[bjzhb666]

📚 The doc issue

In https://docs.vllm.ai/en/latest/examples/offline_inference/vision_language_multi_image.html, I do not find llava onevision, but I think this model supports multi images? Can VLLM do this? thanks

Suggest a potential alternative/fix

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Isotr0py]

Yes, vLLM supports multi images for llava onevision.

[bjzhb666]

But it is not in the document, maybe we should update an example in it. Thanks. Can you share an example?

[bjzhb666]

@Isotr0py Hi, can you give me an example?

[Isotr0py]

Sorry for missing this message. Try this:

from vllm import LLM, SamplingParams
from vllm.assets.image import ImageAsset

images = [ImageAsset("stop_sign").pil_image, ImageAsset("cherry_blossom").pil_image]
prompt = f"<|im_start|>user <image>\n<image>\nWhat do you see here?<|im_end|> <|im_start|>assistant\n"

llm = LLM(
    model="llava-hf/llava-onevision-qwen2-0.5b-ov-hf",
    max_model_len=25600,
    enforce_eager=True,
    limit_mm_per_prompt={"image": 2},
)

sampling_params = SamplingParams(temperature=0.2, max_tokens=128)
inputs = {
    "prompt": prompt,
    "multi_modal_data": {"image": images},
}

outputs = llm.generate(inputs, sampling_params=sampling_params)
for o in outputs:
    generated_text = o.outputs[0].text
    print(generated_text)

[bjzhb666]

Thanks for your response