feat: improve message content chunks handling

[drbh]

This PR improves the parsing of content in messages by deserializing all content into ContentChunks (a vector of ContentChunk). Each ContentChunk is an enum representing either raw text or an image URL. For minimal change impact, ContentChunks can be serialized directly into a flattened string.

[drbh]

**note this PR improves how images are handle when sent as separate content in the chat endpoint. If ![]() markdown images are included they are still handled the same way, and a future RP may deprecate markdown images in chat and improve how images are sent to the model

example request

from huggingface_hub import InferenceClient

client = InferenceClient("http://127.0.0.1:3000")

chat = client.chat_completion(
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Whats in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png"
                    },
                },
            ],
        },
    ],
    seed=42,
    max_tokens=100,
)

[Narsil]

Can you add some tests too for the deserialization ?

[drbh]

updates:

this pr now also parses markdown images into typed ContentChunks

for example both the string and structured JSON inputs are deserialized into ContentChunks and both currently serialize back into a string

text-generation-inference/router/src/lib.rs

Lines 1277 to 1288 in f8be8d5

	let content = json!("Whats in this image?");
	let chunks = message_content_serde::deserialize(content)
	.expect("Failed to deserialize")
	.unwrap();
	assert_eq!(
	chunks,
	ContentChunks(vec![
	ContentChunk::Text("Whats in this image?".to_string()),
	ContentChunk::ImageUrl("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png".to_string())
	])
	);

and

text-generation-inference/router/src/lib.rs

Lines 1300 to 1319 in f8be8d5

	let content = json!([
	{"type": "text", "text": "Whats in this image?"},
	{
	"type": "image_url",
	"image_url": {
	"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png"
	},
	},
	]);
	let chunks: ContentChunks = message_content_serde::deserialize(content)
	.expect("Failed to deserialize")
	.unwrap();
	assert_eq!(
	chunks,
	ContentChunks(vec![
	ContentChunk::Text("Whats in this image?".to_string()),
	ContentChunk::ImageUrl("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png".to_string())
	])
	);

[Narsil]

This is exactly what we do not want.

[drbh]

closing in favor of #1906