Skip to content

Notes for PR to main llama.cpp: server tools: JSON subprocess commands

Jump to bottom
Emmanuel FARHI edited this page Aug 7, 2026 · 5 revisions

What may be improved

The current tool set is, factually, a white list. It contains the shell execution which is open.

Any new tool requires a PR and specific C code in the server-tools.cpp file. Hard-coded tools are secure by construction (hopefully) except open-shell. Some of them, still, call a shell to execute a command. In the end, the server-tools file gets bigger and bigger.

The exec_shell_command tool is very handy, but also widely open, as it allows any command execution, including command injection. There are no security control mechanism (and in a way this is a will so that we fear this is dangerous). This is not satisfactory on a production server. The risk arises from the way this tool is implemented: it calls a shell with a free text command passed as is and interpolated, e.g. "sh -c 'command'. This is very dangerous in my view. The pure-IT solution is to isolate the LLM or an MCP, which adds a lot of complexity in my view. And still this is not enough. I think there are simpler solutions with better threat control.

Requirements

I need ways to control the type of commands which are executed on the server, in a strict and simple way. Currently there are different solutions:

1- add more hard-coded tools in server-tools.cpp (e.g. #25875 #24469 #26522 #25877 #25848 #22132 ): this adds more code to maintain, and the list will continuously grow as we face new requirements for commands.

2- add filters to control the type of commands to execute (e.g. #23956 #24659): filters are never exhaustive enough, and they give false confidence.

3- isolate the execution environment: Adding a Docker/Podman container for execution is certainly a wise solution, but it adds more complexity, and does not entirely avoid command escalation. In the box, the LLM is given full capability to exploit all leaks/CVE and escape. In case there is a mounted area, it can (and will) be wiped out. And when given network access, the AI will execute remote tasks (e.g. curl, playwright/selenium, etc). In my view, this is not enough: a better security control is required at the tools level, e.g disable the exec_shell_command and replace it by a white list of chosen commands that correspond to the server use-case.

4- use external MCP servers: this also adds more complexity as opposed to a flexible tool-set directly controlled by the llama-server. This is the solution presented in discussion #26036.

Solution

I recommend a definitive solution, which gives full freedom in the tools definition, without actually hard-coding them as it goes through a configuration file. This is a continuation from #24659 but with a strict control of the executed command, no shell and interpolation, no command injection, no need for filters.

The idea is to execute commands directly by subprocess without shell. The commands are initialised by reading configuration files which hold descriptions and commands. The commands do not call a shell (except when explicitly part of the command spec - and we should know about the risk - see examples below), and are passed as is for execution into a subprocess. Some placeholders allow string replacements from the context, but as static strings. There is no string interpolation, and the command structure can not be changed. All arguments are passed as fixed strings.

The configuration information is inserted into the list of tools (server_tools::setup), and each tool appears as if it would have been hard-coded.

Regarding the configuration file format, I've experienced JSON and Markdown formats: both are easy to handle. The configuration file can be discussed. Currently, it has to be adapted to e.g. Windows/Linux systems, but I bet we could think of ways to define commands for both systems. Typical tools configurations can also be given in the examples/.

As an example, let's have a look at how we would tackle the existing tools and possibly handle further requirements from discussions/issues/PR. I guess you will have remarks concerning the example table below. Keep in mind these are just examples, but the intent is to show that we can get rid of some hard-coded tools, and just set a mechanism to configure tools via simple files.

tool current implementation Example command in Configuration file (linux)
read_file io->read_file / rdbuf cat {filename}
file_glob_search io->list_entries find /search/path/ -maxdepth N -type f -name "pattern*" -exec grep -l "{pattern}" {} \;
grep_search regex_search grep -r {pattern} {path}
exec_shell_command "sh", "-c", command sh -c {command} (as dangerous as exec_shell_command)
write_file io->write_file f << content echo {content} >> {file}
edit_file apply_replacements sed -i 's/{old_text}/{new_text}/g' {file}
get_datetime gmtime strftime date +{format}
get_info "uname", "-a" uname -a
web_search none a ddgs based script, same as ddg script in #26036

For instance a TOOLS.json` configuration file could hold:

[
  {
    "name": "get_ip",
    "description": "Get the public IP address of the server",
    "command": "curl ifconfig.me"
  },
  {
    "name": "list_files",
    "description": "List all files in a {directory}",
    "command": "ls -la {directory}"
  },
  {
    "name": "date_now",
    "description": "Get the current time with given {format} which defaults to '%a %b %e %H:%M:%S %Z %Y '",
    "command": "date +{format}"
  }
]

and we would enable it with e.g. llama-server --tools TOOLS.json. Such configurations come on top of hard-coded enabled tools.

So this solution could even replace most existing hard-coded tools, and strip down a lot the server-tools.cpp file, but I personally think we should still keep the existing tools.

This suggestion is not highlighted as a 'secured' exec_shell_command. It is a solution to better handle the server-tools in a flexible way.

I think the MCP solution #26036 is complementary to the solution I propose. The MCP in a container (as stated in #23956) can be considered as a secured environment, but more complex to set up. And in fact it just moves to risk from the LLM side to the MCP.

Files: JSON version

Files: Markdown version

build:

  • cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DLLAMA_OPENSSL=ON -DGGML_CUDA=ON
  • cmake --build build --config Release -j$(nproc)

test:

  • build/bin/llama-server -hf ggml-org/gemma-4-E4B-it-GGUF:Q4_0 --tools TOOLS.json

Clone this wiki locally