Added Mllama

[tgaddair]

See: https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct

Based on TGI implementation here: huggingface/text-generation-inference#2568

Usage:

from lorax import Client


endpoint_url = "http://127.0.0.1:8080"
client = Client(endpoint_url)

image = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
prompt = f"""<|begin_of_text|><|start_header_id|>user<|end_header_id|>

![]({image})If I had to write a haiku for this one, it would be: <|eot_id|><|start_header_id|>assistant<|end_header_id|>"""

for resp in client.generate_stream(
    prompt,
    max_new_tokens=50,
    temperature=1,
):
    if not resp.token.special:
        print(resp.token.text, sep="", end="", flush=True)

[arnavgarg1]

Amazing