GLM 4.7 leaking interleaved thinking tokens

[garymjr]

I noticed with GLM 4.7 using the z.ai coding plan that interleaved thinking seems to leak into tool calls and causes the agent to stop. Here's an example of what this looks like.

Let me continue exploring the remaining packages to get a complete picture. </thinking><tool_call>read<arg_key>path</arg_key><arg_value>/Users/gamurray/Personal/pi-mono/packages/agent/README.md</arg_value></tool_call><tool_call>read<arg_key>path< /arg_key><arg_value>/Users/gamurray/Personal/pi-mono/packages/coding-agent/README.md</arg_value></tool_call><tool_call>bash<arg_key>command</arg_key><arg_value>ls -la /Users/gamurray/Personal/pi-mono/packages/agent/src/</arg_value></tool_call><tool_call>bash<arg_key>command</arg_key><arg_value>ls -la /Users/gamurray/Personal/pi-mono/packages/coding-agent/src/</arg_value></tool_call>

I have a fix that seems to be working locally, but I'm running it a little longer to make sure it's the correct solution. Not sure if anyone else has seen this and started working on a solution as well.

Thanks for pi, I'm really enjoying the experience over other agents.

[aliou]

Hello! Noticed the same issue with gemini-3-pro via google (see simple repro in session file).

I have a ugly fix that works locally, but only for google.

I can't tell if this is a temporary issue because the model is still in preview, but seeing that it happens for other model it might be worth harmonising how this is handle? wdyt ? :)

[garymjr]

Interesting! Sure, here's what I'm running locally. I haven't seen the thinking leak again and I've been using this branch all day.

[aliou]

@garymjr spent some more time investigating and iterating and i basically ended up with the same changes as yours :) Do you think you could open a pull request? Thanks!

[aliou]

The annoying thing tho is that this is a model issue; it kinda feels weird having a workaround for it? Both models are relatively new so it might be fixed later, but then again the thinking issues for Gemini has been there since day one (cf. this tweet mentioning it).

Thoughts @badlogic @nicobailon ?

[aliou]

ah just noticed this pr #302 that is another take on this

[badlogic]

@garymjr GLM uses the anthropic provider code, which was just fixed in 29379ea by converting unsigned thinking blocks to plain content blocks when submitted to the LLM. This affects thinking blocks in prematurely aborted LLM responses only, so a conversion to plain content blocks is fine.

For Gemini, I'm not sure if we want to fix this on our end. That's something big G has to fix in their endpoint I think. I'm not bullish on hacking around issues in providers, as that will make our codebase harder to maintain, especially since those edge cases are hard to test. @aliou 's repro above for example doesn't repro for me at all after 10 tries (using both straight Google gen-ai endpoint, and Antigravity OAuth endpoint).

@garymjr I also saw you made fixes to the openai-completions code. Can you explain those? Which model/provider was this necessary for? GLM doesn't use that code.

[garymjr]

GLM has both anthropic and openai-compatible endpoints. Admittedly I don't know if the openai changes are necessary since I'm using the anthropic endpoint as well.

[badlogic]

In pi, we only use the Anthropic messages implementation for talking to Anthropic endpoints. openai-completions works with many providers, chaning its behaviour is not necessary at this point. The fix for Anthropic is live in 0.27.7. Thanks for reporting!

[garymjr]

Awesome! Thanks for the quick fix.

[garymjr]

@badlogic even after 29379ea i'm still seeing thinking blocks leaking into tool calls and causing the agent to stop working. This still seems to work though. cc @nicobailon @aliou

[badlogic]

It's an issue with z-ai. If we do what you suggest for Anthropic, things just break.

z-ai swallows the thinking block without signature.

I can't get z-ai to spit out with the currently commited approach. Do you have a repro? Maybe a session JSONL where it happened? Can send to contact@mariozechner.at

[garymjr]

@badlogic just sent you a session file. It looks like what's happening is a tool call inside a thinking block is killing the session. I could be wrong though. Still wrapping my head around some of this.

[saeedesmaili]

[Sorry, unrelated to the topic, will delete it later]
Curious how are you using z.ai's coding plan with pi? I don't see the option when using /login command. OAuth providers I see are only: anthropic, github-copilot, google-gemini-cli, google-antigravity

[aliou]

pi looks for the ZAI_API_KEY, I think /login is mainly for providers with oauth flows. (I might have missed wether z.ai also as a oauth flow though)

[badlogic]

That's right. Should add login for known providers as well I suppose, which just saves out the api key to auth.json.

…

On Fri, Dec 26, 2025, 14:52 Aliou Diallo ***@***.***> wrote: pi looks for the ZAI_API_KEY, I think /login is mainly for providers with oauth flows. (I might have missed wether z.ai also as a oauth flow though) — Reply to this email directly, view it on GitHub <#292 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD5QBGENR7HCGAR3JNUGB34DU4RRAVCNFSM6AAAAACP4BPLXKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKMZUG4YTEOI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[garymjr]

@badlogic trying a new change, not sure if it's better or worse, but it's correctly processing the tool calls in the thinking blocks from zai. this is based on that latest main so it doesn't have my previous attempt in it.

[garymjr]

this seems to be fixed by 04a7647. closing this now. thanks all

[badlogic]

Thanks for your help!

…

On Tue, Dec 30, 2025, 10:01 PM Gary Murray ***@***.***> wrote: this seems to be fixed by 04a7647 <04a7647>. closing this now. thanks all — Reply to this email directly, view it on GitHub <#292 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD5QBHZJRE66MVL3KBPEO34ELRY7AVCNFSM6AAAAACP4BPLXKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKMZXG42TKOA> . You are receiving this because you were mentioned.Message ID: ***@***.***>