Intermittent toolcall failures

[unleashit]

While testing out open terminal, I've been having this issue pretty consistently. After requesting some changes at a context of around 30,000 (plus or minus), llama cpp server sometimes goes idle in the middle of a replace_file_content tool call. It's always after both the input/output is fully prepared and displayed in the tool window but before any change to the file system made. It works more than it doesn't (fails maybe 1 out of 5 times), but it's making it pretty much unusable. Ending the process, then continuing allows it to move on "sometimes", but then the problem usually happens again within the next several tool calls.

Same results with 3 different models: Qwen3.5 122b, 35b a3 and Devstral Small 2 24b, all with native tool calling selected in openwebui. Using other coding tools like open code or roo using the same models and model server are problem free.

Openwebui is 0.8.10, Open Terminal 0.11.26 and Llama.cpp 8467 (latest build as of today).

[unleashit]

I just updated the title because after a bit more testing I see that it's not just the replace_file_content tool. Other tools like list_files, read_file and more sometimes won't complete.

I also noticed that the connection to open terminal seems fragile, and I occasionally get this in the open terminal log: open_webui.utils.tools:get_terminal_tools:974 - Terminal server not found: http://192.168.x.x:<port>. I had some issues for a few minutes in the UI once too, but after restarting the containers it worked again.

[iChristGit]

What is the context length that you’re using?

I’ve been also messing around with it and it seems pretty reliable overall.

Tool calls that require long text like replace file content or create file to take more time but they finish

[unleashit]

@tjbck much respect for your work on this project and openwebui, but converting this to a discussion without even a comment is pretty much the type of thing that ended Stack Overflow. No doubt you're busy and I can see both sides of the problem especially these days with the onslaught of bots and vibe coders. But I'm not one of those and frankly this is probably the last time I'll bother to file an issue on an open source project I'm not able to dig very deeply into because of time. Maybe you think that's the way it should be, but the side effect of that standard is to cut off a lot of potentially good data (not to mention discourage possible future participation).

@iChristGit I haven't seen the issue either with simple queries and a few tool calls. It always began at a medium context around 30,000-ish, after 10 or so tool calls. Might be worth adding that all the sessions had images uploaded through the openwebui UI and involved multi-modal models, but it seems likely the problem relates to the flaky connection with open terminal I mentioned.

[iChristGit]

My test were up to 80000 tokens as this is where my VRAM gets exhausted.

Can you give example of a task that you're trying to achieve and I'll try that myself and see if it hangs out in any stage

[unleashit]

Pretty lucky here with 128gb :) The Qwen3.5s are set to the max, which is around ~262,000 tokens. So I don't think it's that.

Sure, but I don't have one that's very straightforward since I was mainly toying around with it and seeing if I could get a somewhat "open code in a browser" experience. I was able to get it to create a custom weather widget with live open mateo data and deploy it to Vercel, then write a skill. I still have that chat, and can see the problem happened once. Maybe I'm expecting too much, but this prompt gave me a lot of trouble:

Create a new next.js app  in the projects folder called trivia. This should be an app that asks the user a series of 10 pop culture trivia questions and grades them at the end. Make it modern and sleek with some nice animations. There should be a progress tracker to show how far along they are. Please do not simply use the same questions each time. There is a free api dedicated to trivia: https://opentdb.com/api_config.php which you should use. When finished, deploy it to vercel using the deploy_to_vercel skill.

It got stuck several times, but eventually it wrote a working app, just with a few issues like bad CSS due to it not being aware of recent Tailwind changes. Next I gave it a screenshot and asked it to try and fix, and it iterated maybe 10 times before freezing again. At that point, I couldn't get it to continue including after starting a fresh session with just the screen shot and a prompt.

I'm fairly confident the issue isn't my setup and this is a bug, but of course I can't be 100% sure. I think I'll give it time and before I try again, as my needs are mostly met with Opencode, etc.

[unleashit]

Update: the tool calling and terminal connection breakages appear to be fixed. The same prompts/models no longer have this issue even with dozens of tool calls. There are response quality issues that may or may not be fixed with an improved system prompt, but is appears open terminal and open webui updates have fixed this issue.