fix(llama.cpp): handle corner cases with tool content

[mudler]

Description

This PR fixes:

                                                                                                                     21:53:42 [23/1877]
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout [PREDICT] Received result: {
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "stream": false,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "cache_prompt": false,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "n_predict": -1,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "top_k": 40,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "top_p": 0.949999988079071,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "typical_p": 1.0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "temperature": 0.8999999761581421,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "repeat_last_n": 0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "repeat_penalty": 0.0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "frequency_penalty": 0.0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "presence_penalty": 0.0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "mirostat": 0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "mirostat_tau": 5.0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "mirostat_eta": 0.10000000149011612,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "n_keep": 0,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "seed": 374377266,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "ignore_eos": false,
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "embeddings": "",
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "correlation_id": "",
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout   "stop": []
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stderr srv       Predict: No tools found in data - tool calls will not work without tools field
9:53PM DBG GRPC(Qwen_Qwen3-0.6B-GGUF-127.0.0.1:46437): stdout }
9:53PM INF HTTP request method=POST path=/chat/completions status=200
time=2025-11-20T21:53:42.716+01:00 level=WARN msg="Error asking LLM, retrying" attempt=6 error="error, status code: 500, status: 500 Internal Server Error, message: rpc error: code = InvalidArgument desc = Value i
s not an array or object: null at row 22, column 40:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n
     ^\n    {%- set tool_end = \"</tool_response>\" %}\n at row 22, column 5:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n    ^\n
  {%- set tool_end = \"</tool_response>\" %}\n at row 18, column 37:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n
                  ^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 18, column 1:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in mes
sages[::-1] %}\n^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 1, column 1:\n{%- if tools %}\n^\n    {{- '<|im_start|>system\\n' }}\n" source.file=/home/mudler/_git/go/pkg/mod/github.com/mu
dler/cogito@v0.5.1/llm.go source.L=32
9:53PM DBG Sending event: {"type":"error","message":"Failed to execute tools: failed to evaluate next action: failed to select following tool: failed to pick tool: failed to get reasoning: error, status code: 500,
 status: 500 Internal Server Error, message: rpc error: code = InvalidArgument desc = Value is not an array or object: null at row 22, column 40:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set
start_of_message = message.content[:tool_start_length] %}\n                                       ^\n    {%- set tool_end = \"\u003c/tool_response\u003e\" %}\n at row 22, column 5:\n    {%- set tool_start_length =
 tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n    ^\n    {%- set tool_end = \"\u003c/tool_response\u003e\" %}\n at row 18, column 37:\n{%- set ns = namespace(multi_s
tep_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n                                    ^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 18, column
1:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 1, column 1:\
n{%- if tools %}\n^\n    {{- '\u003c|im_start|\u003esystem\\n' }}\n"}

Notes for Reviewers

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	1ee48ac
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/691f920c0a1f9d0008c2a8ef
😎 Deploy Preview	https://deploy-preview-7324--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[mudler]

error looks unrelated, as it seems to hit master nonetheless. last green test on apple was from 5fed9c6