server: thinking type rejected as invalid but used by Ministral 3

[isaac-mcfadyen]

EDIT: Apologies for marking this as a bug, probably should have marked as an enhancement - feel free to re-tag.

Name and Version

❯ ./llama.cpp/build/bin/llama-server --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes
version: 7236 (a2b0fe8)
built with cc (Gentoo 13.4.1_p20250807 p8) 13.4.1 20250807 for x86_64-pc-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server --port 8000 -fa on -c 8192 -ngl 99 -cram -1 -m /models/Ministral-3-14B-Reasoning-2512-UD-Q6_K_XL.gguf

Problem description & steps to reproduce

The new Ministral 3 reasoning models use a (I believe non-standard) content[].type of thinking which is used to pass the reasoning history traces back to the model:
https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512/blob/main/chat_template.jinja#L94

However, llama-server currently rejects this with an error:

srv    operator(): got exception: {"error":{"code":500,"message":"unsupported content[].type","type":"server_error"}}

Is this something that should/could be fixed on the llama-server side? Or is there a more standardized way of passing reasoning traces back in that e.g. we should encourage people converting the Magistral 3 GGUFs to add into the chat template?

First Bad Commit

N/A, not a regression

Relevant log output

[eauchs]

idk

[eauchs]

Yes, this is a limitation in the current OAI compatibility parsing in llama-server. It explicitly throws an exception on content types other than text or image_url.

To support this properly in llama.cpp, we need to patch the JSON parsing logic (likely in server.cpp or associated OAI utils) to accept type: "thinking". It should be handled effectively as text content so the reasoning trace passes through to the model's chat template without error.

Workaround: For now, you can modify your client/middleware to rename thinking to text in the messages content array before sending the request to the server.

[isaac-mcfadyen]

Workaround: For now, you can modify your client/middleware to rename thinking to text in the messages content array before sending the request to the server.

The issue with this is that it won't insert the reasoning trace properly unless the reasoning tags are manually added to the text, correct?

E.g the chat template adds thinking content between [THINK] and [/THINK] but if the client sends it as part of a text content block it won't be put between those tags.

[eauchs]

Correct.

For the client-side workaround: You would indeed need to manually wrap the content in the specific tags (e.g., [THINK]...[/THINK]) and send it as text.

For the actual server-side fix: Simply treating it as text internally would likely fail as you described. The llama-server implementation needs to accept the thinking type during parsing and pass it effectively to the templating engine as a distinct type, so that the Jinja template logic (which checks if message['type'] == 'thinking') can trigger and insert the correct tags automatically.

[aldehir]

Other models accept reasoning traces back in the reasoning_content field and map it internally to the appropriate field in the template. I don't think this should be any different.

[isaac-mcfadyen]

CC @danielhanchen (hope you're OK with the ping!) as someone who's created GGUFs for these models.

I know you've done fixes for chat templates of other models in the past, is this something that could be considered for Ministral 3?