Allow models to set title using a native tool call.

[DanielHe212]

This seems like a bit more work than what belongs in an issue since this would likely require changes in multiple areas of code, but it would address a small annoyance I've had for a while.

I run openwebui and llama.cpp locally, and letting the model generate title causes recomputing the entire kv cache after the first chat turn. This would not be a huge issue, except the tasks I give to my models often lead to a pretty big kv cache after the first chat turn. Therefore I turned off generating titles and the chat titles simply come from the first line of my first message.

But these titles are less descriptive, and I do wish I had more descriptive titles.

I've noticing nowadays that modern models like Qwen3.5/3.6 have very good native tool calling capabilities and aren't pushed off track easily, and they can follow system prompt instructions very well. I'm thinking it could be useful to have a native tool that sets the title of the chat, and put an instruction in the system prompt to tell the model to use it. Most of the time, the title can be accurately set based off of the user's first message anyway, so the title can be set by a tool call from the model immediately, before it even does anything else.

This would solve the huge computational cost of the previous way of setting chat titles.

[castielwaverly]

I've started considering running a second, smaller model offloaded to RAM, specifically for title generation, to get around this. llama.cpp has a setting for multiple cache slots (i.e. -np N or --parallel N), which seems to preserve cache across different chats, but not for title generation specifically, for some reason. This does seem to be a case of both tools working "as expected", but perhaps not talking to each other properly 😆.

In general, I think the problem of KV cache getting wiped for title generation does not necessarily need its solution space narrowed down to a rework of how background tasks are run, as there are likely ways to make these tools work together. I actually scrolled past this discussion at first while looking for other people facing the problem, and only came here on a second pass. I wonder if it's the case that not too many people have this problem or if the people who do just don't realize this discussion is about that.