Searchtool won't work

[Hetare-byte]

Hello, i managed to setup odysseus through docker and downloaded some models. The problem is that none of them seel to succesfully be able to use the search tool. I will link the models i've tried and also show some screenshot examples of responses i get when asking a simple question such as what the current weather is like in brussels. I have spent hours trying to fix it but just can't. I have a 2070 super. Is it possible that these smaller models just are not able to use tools?

Thanks for the help in advance!

[ooovenenoso]

Yes — from the screenshots this looks more like a model/tool-calling limitation than a broken SearXNG config.

Your Web Search settings show SearXNG active at http://host.docker.internal:8080 with DuckDuckGo fallback, so first click Test there. If the provider test succeeds, the search backend is probably fine.

The issue in your chat examples is that the local models are not actually calling Odysseus’ web_search tool. They are improvising instead:

• mistral:7b-instruct tries to write a Python/OpenWeather script.
• llama3.1:latest talks about loading skills / bash rather than making a clean search call.

That usually means the model is not following the tool-call format reliably. Smaller 7B/8B local instruct models can answer chat fine but still be weak at agent/tool use, especially on tight VRAM.

Things to try:

1. In Settings → Web Search, press Test and confirm SearXNG returns results.
2. In the chat bar, make sure Agent mode and the search/tool option are enabled.
3. Try a model known to follow tool/function-call formats better. Qwen instruct/coder models are often better for this than older Mistral/Llama 7B-class models, though it still depends on the exact template/serve backend.
4. Check the server logs around the failed message. If you see no web_search invocation at all, it is model-format/compliance. If you see a web_search invocation followed by provider errors, then it is search-provider config/networking.

So yes, it is very possible the smaller models you tried simply are not reliably using tools. The key distinction is: search provider test fails = SearXNG/network config issue; model never calls web_search = model/tool-call formatting issue.

[Hetare-byte]

Hello, thank you for the fast respons!

As i was about to test these things you suggested, i must have f'd something up and now none of my models are responding. They all just generate empty text boxes. I am really sorry for the trouble but i've spent quite some time trying to fix this new issue to no avail. Would you happen to know what i messed up and how i reverse it? Apologies if these type of questions are not what this forum is for, if it's this is not the right place to ask then i'll drop it and give odysseus another shot some time later eventhough i'm really excited to use it.^^

Thank you in advance!

[Hetare-byte]

Okay somehow the models are working again!

I performed the tests you mentioned.

1: I tried testing SearXNG and the result comes back positive.

2: I made sure to actate agent mode and have search enabled.

3: chatGPT recommended me to use these models to test and claimed they were better at tool use, but they also failed.

And i'm not too sure where i see the logs, sorry i'm very new haha. Thanks for the help though!

[Jelleus]

I am also currently testing different models to see how well they work with the given tools. I started with Qwen 3.5 9B and ended with Qwen 3.6 27B. In case of Qwen 3.5 9b which I suppose "most" people could run, I found no problems with tool usage and it generally makes good answers. Even deep research worked well enough with it. But obviously it is still only a 9b model, so for example, even if it can quite reliable use the web_search tool, it sometimes just does not make the best search queries. But for people with limited Vram, I would strongly suggest the Qwen 3.5 models.

Best of luck.

[Hetare-byte]

I'm glad to hear that you didn't have a problem with tool usage for Qwen3.5 9B, that means that i am capable of running a model with tool usage. But on the other hand this does mean that my problem lays elsewhere and not in the model itself. Is there any screenshots of anything i could for people with more klnowledge then me to look at and maybe spot the problem? Countless of hours spent on chatGPT and claude trying to get it fixed lead me nowhere haha

[y6htr6w568-eng]

I'm having the same issue as you and I'm getting different results. For example deepseek-r1:8b managed to somehow search the web however, before i could give me an answer it gave me this error: ( host.docker.internal returned HTTP 400: {"error":{"code":400,"message":"request (4304 tokens) exceeds the available context size (4096 tokens), try increasing it","type":"exceed_context_size_error","n_prompt_tokens":4304,"n_ctx":4096}} ). Meanwhile other models such as deepseek-v2:16b
will flat out refuse to give me an answer and insist they do not have access to the web.

[Hetare-byte]

I managed to get the smaller models like Qwen3.5 and gemma12b to perform proper web searches. ChatGpt suggested i'd make a persona and inject this in their terminal

"When you need to search the web for current information, prices, news, or anything requiring live data, you MUST use this exact format:

your query here

This is the ONLY correct way to search the web. Never use curl, bash, python, or any other method to retrieve live web data. The web_search fenced code block executes automatically and returns results directly.

Example — if asked for Bitcoin price:

bitcoin price USD today

Wait for results, then summarize them clearly."

And that actually did the trick. It feels a bit forced this way but it works, My other tools aren't functional though. And eventhough they are able to perform websearches now, they still are wrong now and then. Ex: i ask the current weather in brussels. And it checked 5 sources, 4 of which were about brussels and 1 was about georgia which isn't close at all, but this may be a limitation of the model itself.

Eitherway i saw in Issue #2724 that Felix would like to try and get odysseus to run better on smaller models which i am very excited for!

[y6htr6w568-eng]

I got a quick question sorry Im a bit unclear. When I copied what you had pasted it worked perfectly. When I tried a new chat myself with a new question it didnt. What did you do and what do you mean by make a persona and inject it? Sorry very new to this.

[Hetare-byte]

All good i'm also very new to it haha, i'll show you with some screenshots

In this image i'm not using a persona, so it fails to use the searchtool.

When i go to a new tab, i click the little arrow up sign in the bottom left corner of the prompt box and go to persona where I have input the instructions i mentioned in the previous post in the persona's System promps As in this image:

And in this following image, i ask it pretty much the same question while using the persona. And it uses the searchtool flawlessly.

That being said though, this is not how it's intended to work and i am still unsure why none of my tools work without extra steps like this. My models can't do anything in my agenda, inspect pdf's i provide or anything of that nature. All this fixes is the fact that they can use the searchtool now. I'm very interested why they refuse to do it properly without these instructions though

[y6htr6w568-eng]

Thank you so much you have been very helpful. I'd like to say that some models do work on their own but they are heavier. Gemma 4 12B has been woking for me at the very least with agent mode and no need for the method you have come up with.

[Hetare-byte]

I'm glad i could help! I hope we'll see some optimization for the smaller models eventuallty ^^