Messages API is missing a whitespace after data: to be compliant

[philschmid]

System Info

ghcr.io/huggingface/text-generation-inference:2.0.4

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

1. Start container
2. run curl request with streaming

Expected behavior

TGI

data:{"id":"","object":"text_completion","created":1717586509,"model":"meta-llama/Meta-Llama-3-8B-Instruct","system_fingerprint":"2.0.4-sha-f426a33","choices":[{"index":0,"delta":{"role":"assistant","content":"Here"},"logprobs":null,"finish_reason":null}]}

vLLM

data: {"id":"cmpl-a33baad7fc1b46518194d433bd046551","object":"chat.completion.chunk","created":1717586822,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":""},"finish_reason":"stop"}],"usage":{"prompt_tokens":23,"total_tokens":68,"completion_tokens":45}}

OpenAI

data: {"id":"chatcmpl-9Wj3xCu6xZ2GlmuQ3hKQuAeeNki4C","object":"chat.completion.chunk","created":1717586745,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_319be4768e","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

[drbh]

Hey @philschmid this should be resolved in the main

The issue was originally related to axum. In versions before 0.7.0 there was no space added after the data: field. This PR fixes it and should be available tokio-rs/axum#2149

As of last week TGI bumped axum to 0.7.0 #1959 so we should see the space in out responses now