JSON schema conversion error in llama.cpp

[ThomasBuchinger]

I've been debugging a problem with kubernetes-mcp-sever and llama.cpp. The Short version is, that the pattern field in JSON-Schema must include the start/end anchors (^regex$) or llama.cpp will throw an error.

The error occurs on parsing the request. Therefore just having kubernetes-mcp-server enabled causes all requests to the LLM to fail. I think the easiest solution is to just add start/end anchors to all patterns, since there does not seem to be a way to disable this behavior in llama.cpp (also affects Ollama and LMStudio).

Problem Description

The MCP server itself works fine. The Agent can list the tools and include the list of MCP-Tools in the request to the LLM. However the Request fails with the following error message:

POST /v1/chat/completions 127.0.0.1 400
{"error":{"code":400,"message":"JSON schema conversion failed:\nPattern must start with '^' and end with '$'","type":"invalid_request_error"}}

llama.cpp internally converts the JSON schema provided by MCP into a grammar, that it checks the output against. (Code in llama.cpp), that conversion fails if the regex doesn't have the start/end anchors and as far as I can tell, there is not option to disable this.

Solution

I've tested, that adding ^$ to every pattern using curl and confirmed, that this will fix the issue. I also checked the existing patterns and it should be fine to add the anchors to them.

I can look into creating a PR for this sometime this week

Workarounds

If you run a Proxy (Like Envoy-AI-Gateways MCPRoute or LiteLLM) that can filter tools from the MCP, you can disable the affected tools

Affected Tools (in the default tools)

• nodes_top
• pods_list
• pods_list_in_namespace
• pods_top
• resources_list

[manusa]

Relates to #277