Feature Request: Do we want web search as part of llama-server

[Swiftyos]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Add two new built in tools for llama-server:

1. Web Search - Find web pages
2. Web Retrieval - Get there content

Motivation

llama-server has become my goto ui for using local llms. It does not have web search or the ability to retrive web pages which limits there ability to have up to date knowledge.

The current solution is to use an mcp server to add that ability, which is a bit fiddly to get setup.

The objective is to have easy seemless access to the web for your localllm.

Note: I have searched issues and discussions and found any in-depth discussions about this.

Possible Implementation

I have created a proof of concept of this capability which adds web search following the same patterns searxng uses with multiple engines supported and a web page retrival tool.

The question is - is this wanted in the llama-server?

If so I will work on getting the solution to a high quality and make a PR.

[skidd-level-100]

Id love it, cant speak to technical/implementation details because I'm just a skid for now.

[ngxson]

it's more suitable to be a frontend-only tool, so that it's easier to configure (e.g. select provider(s), api key, etc)

the wget tool can already use the CORS proxy endpoint in llama-server so I don't think there is any code should be added to llama-server

this seems like a llama-ui feature, not sure if @allozaur already had this in the roadmap ?

[allozaur]

Yeah, i think we want to do this on the frontend only and probably allow to use existing providers (like Exa) or a custom mcp server

[lollo78]

Hi everyone,
I’m using the latest release of llama.cpp on Windows and recently noticed the built-in MCP (Model Context Protocol) integration in the SvelteKit WebUI proxy.

It's amazing to see how seamlessly the UI handles tools like web_search_exa and huggingface out of the box.

When I toggle them on using the UI selector, they work perfectly during the session (the model triggers web_search_exa correctly, handles permissions, and executes the search).

However, I've run into a minor usability issue: the enabled MCP servers disappear every time the browser cache/localStorage is cleared or when the backend restarts.

Since these servers seem to be managed on-the-fly by the built-in Node/npx proxy architecture of the WebUI, I would like to know what the canonical way is to make them permanent at startup from the backend side.

1. Is there an officially supported environment variable or a specific flag configuration (like --mcp-config) tailored for the native WebUI proxy setup?

2. If a JSON configuration file is supported natively by the server binary without third-party environment wrappers, what is the exact schema expected by llama-server to autoload these tools?

I’d love to automate this via a startup script/preset rather than manually re-enabling the toggles in the browser every time.

Thanks in advance for your help and for the incredible work on llama.cpp!

[allozaur]

Hi everyone,
I’m using the latest release of llama.cpp on Windows and recently noticed the built-in MCP (Model Context Protocol) integration in the SvelteKit WebUI proxy.

It's amazing to see how seamlessly the UI handles tools like web_search_exa and huggingface out of the box.

When I toggle them on using the UI selector, they work perfectly during the session (the model triggers web_search_exa correctly, handles permissions, and executes the search).

However, I've run into a minor usability issue: the enabled MCP servers disappear every time the browser cache/localStorage is cleared or when the backend restarts.

Since these servers seem to be managed on-the-fly by the built-in Node/npx proxy architecture of the WebUI, I would like to know what the canonical way is to make them permanent at startup from the backend side.

1. Is there an officially supported environment variable or a specific flag configuration (like --mcp-config) tailored for the native WebUI proxy setup?

2. If a JSON configuration file is supported natively by the server binary without third-party environment wrappers, what is the exact schema expected by llama-server to autoload these tools?

I’d love to automate this via a startup script/preset rather than manually re-enabling the toggles in the browser every time.

Thanks in advance for your help and for the incredible work on llama.cpp!

Yeah. That's sth that could maybe be addressed by providing an .ini file or maybe finally implement the mcp.json support (#20769).

Wdyt @ngxson? Maybe @ServeurpersoCom could take this one?

[allozaur]

But generally I see 2 options:

• mcp.json, but that would also probably need implementing the stdio mcp servers support from the llama serve server-side level
• allow to define the MCP servers config via flag as u suggested above

The latter option allows us to stick to what we currently have, which is client-side only support for remote MCP Servers via SSE/Streamable HTTP/WebSocket connecteions.

[ServeurpersoCom]

The "frontend.json" file, which can be passed via the backend --ui-config-file, allows for the definition of MCP servers. Do we need more user friendly ?

The annoying thing is that the default servers provided are excellent, but if you delete them, you have to delete all the settings just to get them back! Unless you add them to the JSON:

"mcpServers": "[{"id":"LlamaUI-MCP-Server-1","enabled":true,"url":"https://mcp.exa.ai/mcp","requestTimeoutSeconds":300,"name":"Exa Web Search","useProxy":false},{"id":"LlamaUI-MCP-Server-2","enabled":true,"url":"https://huggingface.co/mcp","requestTimeoutSeconds":300,"name":"Hugging Face","useProxy":false}]",

[lollo78]

Yeah. That's sth that could maybe be addressed by providing an .ini file or maybe finally implement the mcp.json support (#20769).

Wdyt @ngxson? Maybe @ServeurpersoCom could take this one?

Thanks for the quick reply!

Implementing an explicit mcp.json support (or expanding the .ini parameters) would be a fantastic quality-of-life improvement for anyone running headless servers or looking for a persistent setup on Windows/Linux without relying on browser state.

For now, I'll keep enabling them manually, but I'm looking forward to seeing how this evolves. Let me know if you need any testing on Windows once a PR or a draft implementation is ready!

[lollo78]

Unless you add them to the JSON:

Thanks a lot! This works as expected to load the servers, but at every startup, the toggles still remain disabled by default.

I have to manually click and enable them every time I launch the server, even though "enabled": true is set in the serialized JSON string. Is there any way to force them to be active on boot?

Code used:
{ "mcpServers": [ { "id": "LlamaUI-MCP-Server-1", "enabled": true, "url": "https://mcp.exa.ai/mcp", "requestTimeoutSeconds": 300, "name": "Exa Web Search", "useProxy": false }, { "id": "LlamaUI-MCP-Server-2", "enabled": true, "url": "https://huggingface.co/mcp", "requestTimeoutSeconds": 300, "name": "Hugging Face", "useProxy": false } ] }

[ServeurpersoCom]

I have to manually click and enable them every time I launch the server, even though "enabled": true is set in the serialized JSON string. Is there any way to force them to be active on boot?

OK. The configuration for this submenu should also be a SYNCABLE_PARAMETER, just like the rest of llama-ui. It requires a little refactor. Actually, that's the only part missing.

[lollo78]

OK. The configuration for this submenu should also be a SYNCABLE_PARAMETER, just like the rest of llama-ui. It requires a little refactor. Actually, that's the only part missing.

Awesome, thank you for confirming that!

It's great to know that the root cause has been identified. I'll manually toggle them for now and look forward to the upcoming releases with this refactor.

Thanks again to everyone for the amazing support!

[ServeurpersoCom]

Yes, this is exactly a follow-up to #25132, where I had already realized I couldn't SYNC the activation of the MCP servers.

[ServeurpersoCom]

So, this "websearch" issue is fine; it's better to put the refactor into a separate feature request. Feel free to open one if you'd like!