[Bug] Tool-call-parser fails to format function calls for GLM5.1 (works for GLM5) in cursor

[MeowGrange]

Checklist

• I searched related issues but found no solution.
• The bug persists in the latest version.
• Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
• If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Describe the bug

When serving the GLM5.1 model , the tool calling / function calling feature does not work correctly through the OpenAI-compatible API.

The sglang server runs fine without any crash or error logs. However, when the model attempts to use a tool (e.g., generating a shell command), the tool-call-parser (presumably the glm47 template parser) fails to recognize and parse the model's output. As a result, the tool call is returned as plain text content rather than a proper tool_calls array in the JSON response.

By using the control variable method, I confirmed that serving the older GLM5 model with the exact same setup works perfectly, and tools are parsed and executed correctly by client applications (like Cursor IDE).

Expected behavior

The sglang tool-call-parser should be updated to support the slightly changed prompt/output format of GLM5.1, so that it can correctly parse the model's raw text into standard OpenAI tool_calls JSON structure.

Additional Context

Client application: Cursor IDE (Cursor fails to recognize the shell tool block when using GLM5.1 via sglang, but works fine with GLM5). Because there are no server-side error logs, it seems to be a pure parsing regex/logic mismatch in the chat template for the new model version.

Reproduction

Server Startup Command:
python3 -m sglang.launch_server
--model-path /workdir/huggingface.co/zai-org/GLM-5.1-FP8
--host 0.0.0.0
--port 30000
--tp 8
--reasoning-parser glm45
--tool-call-parser glm47
--mem-fraction-static 0.85
How to reproduce:

1. Start the server using the command above with a GLM5.1 model.
2. Connect Cursor IDE to http://127.0.0.1:30000/v1 and try to use a Cursor Agent with shell tools.
3. Observable behavior: In Cursor, the shell tool fails to execute normally. Instead of triggering the tool, the model just outputs the raw tool calling syntax as plain text in the chat. This indicates sglang fails to parse the GLM5.1 output into the standard tool_calls JSON format required by the client. (Note: Using the exact same setup with a GLM5 model works perfectly, and shell tools are executed normally in Cursor).

Environment

Python: 3.10.12
CUDA available: True
GPU 0,1,2,3,4,5,6,7: NVIDIA H20-3e
GPU 0,1,2,3,4,5,6,7 Compute Capability: 9.0
NVCC: Cuda compilation tools, release 12.6, V12.6.68
CUDA Driver Version: 550.127.08
PyTorch: 2.9.1+cu128
sglang: 0.5.10.post1

[taegeonum]

Please check sampling parameters. Probably the temperature will be set to 0.0 (this should be 1) if you use openai-compatible APIs.

[MeowGrange]

Hi @taegeonum, thanks for the quick reply and the suggestion!
I think the core issue is not just the sampling parameter.
Here is my control variable test:

1. Client: Cursor IDE (sends the exact same payload/parameters).
2. Backend: sglang serving GLM5.
   -> Result: Tool calling works perfectly. Cursor recognizes and executes the shell tools.
3. Backend: sglang serving GLM5.1 (with exactly the same launch parameters).
   -> Result: Tool calling fails. The model outputs the tool syntax as raw text.

Since the client's sampling parameters (like temperature) are identical in both cases, the fact that it only breaks on GLM5.1 strongly suggests that the tool-call-parser is failing to handle a slight change in the output format or prompt template of the newer GLM5.1 model.

[taegeonum]

Hi @taegeonum, thanks for the quick reply and the suggestion! I think the core issue is not just the sampling parameter. Here is my control variable test:

1. Client: Cursor IDE (sends the exact same payload/parameters).
2. Backend: sglang serving GLM5.
   -> Result: Tool calling works perfectly. Cursor recognizes and executes the shell tools.
3. Backend: sglang serving GLM5.1 (with exactly the same launch parameters).
   -> Result: Tool calling fails. The model outputs the tool syntax as raw text.

Since the client's sampling parameters (like temperature) are identical in both cases, the fact that it only breaks on GLM5.1 strongly suggests that the tool-call-parser is failing to handle a slight change in the output format or prompt template of the newer GLM5.1 model.

There is no issue in tool-call-parser because GLM5 and 5.1 uses exactly the same tool call format. When I tested, GLM5 is robust with temperature 0, but GLM5.1 is not. So please check the sampling params whether the temperature is 0 or not.

[MeowGrange]

Hi @taegeonum, thanks for the quick reply and the suggestion! I think the core issue is not just the sampling parameter. Here is my control variable test:

1. Client: Cursor IDE (sends the exact same payload/parameters).
2. Backend: sglang serving GLM5.
   -> Result: Tool calling works perfectly. Cursor recognizes and executes the shell tools.
3. Backend: sglang serving GLM5.1 (with exactly the same launch parameters).
   -> Result: Tool calling fails. The model outputs the tool syntax as raw text.

Since the client's sampling parameters (like temperature) are identical in both cases, the fact that it only breaks on GLM5.1 strongly suggests that the tool-call-parser is failing to handle a slight change in the output format or prompt template of the newer GLM5.1 model.

There is no issue in tool-call-parser because GLM5 and 5.1 uses exactly the same tool call format. When I tested, GLM5 is robust with temperature 0, but GLM5.1 is not. So please check the sampling params whether the temperature is 0 or not.

1. I checked the final sampling temperature after Cursor returns, and for both GLM5 and GLM5.1, the sglang final sampling temperature is consistently 1.
2. I printed out the messages returned by Cursor and confirmed that the tool execution chain works correctly—Cursor indeed executes the tools and the results are successfully returned to the model.
3. I added a toggle experiment in serving_chat.py that rewrites messages with role=tool to role=user before entering the template. After this change, GLM5.1 successfully processes the returned content in Cursor. This indicates that, in the current combination of sglang + Cursor + GLM5.1, GLM5.1 has compatibility issues with OpenAI-style role=tool messages. The same tool output works correctly if rewritten as a normal user message.

I will continue to investigate to see if I can fix this issue.

[JaredforReal]

@taegeonum @MeowGrange Thanks for the feedback!

1. We have already updated chat_template.jinja in HuggingFace GLM-5.1 chat_template and HuggingFace GLM-5 chat_template
2. fix: normalize tool message content for GLM5.1 chat template #22595 This PR should have solved OpenAI-style tool response issue.

Can u guys apply the PR and use the updated chat_template.jinja file to give it another try? Thanks a lot~

[MeowGrange]

@taegeonum @MeowGrange感谢您的反馈！

1. 我们已经在HuggingFace GLM-5.1和HuggingFace GLM-5chat_template.jinja中更新了 chat_template。
2. 修复：规范 GLM5.1 聊天模板的工具消息内容 #22595此 PR 应该已经解决了 OpenAI 风格的工具响应问题。

你们能不能提交这个 PR，然后用更新后的 chat_template.jinja 文件再试一次？非常感谢！

I have updated my local GLM-5.1 chat_template.jinja file and pulled the latest main branch with the merged PR.
After restarting the server, the issue is completely resolved! The shell tools in Cursor are now working perfectly with GLM-5.1. The model correctly receives the tool response and no longer gets stuck in the infinite retry loop.
Thank you so much for the lightning-fast fix!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.