Tools filtering/sandboxing

[jhrozek]

This is just an idea, feel free to close if it's too out there.

A lot of the agentic assistants use tools. These are passed in the user message, following is an example from anthropic:

 "tools": [
    {
      "name": "builtin_read_file",
      "description": "Use this tool whenever you need to view the contents of a file.",
      "input_schema": {
        "type": "object",
        "required": [
          "filepath"
        ],
        "properties": {
          "filepath": {
            "type": "string",
            "description": "The path of the file to read, relative to the root of the workspace."
          }
        }
      }
    },
    {
      "name": "builtin_create_new_file",
      "description": "Create a new file",
      "input_schema": {
        "type": "object",
        "required": [
          "filepath",
          "contents"
        ],
        "properties": {
          "filepath": {
            "type": "string",
            "description": "The path where the new file should be created"
          },
          "contents": {
            "type": "string",
            "description": "The contents to write to the new file"
          }
        }
      }
    },
    {
      "name": "builtin_run_terminal_command",
      "description": "Run a terminal command in the current directory. The shell is not stateful and will not remember any previous commands.",
      "input_schema": {
        "type": "object",
        "required": [

It would be nice to be able to define a policy or sandbox the tools with codegate. Some examples might include:

• I don't want to allow any tools that read files to read files under db/schema
• I never want to allow any tools that communicate over the network

Because the tools are generic JSON schema I think we have two options:

• either have allow list/deny list of tools or their arguments. We would have to tailor this to the tols and constantly update. Ugh.
• (inspired by Luke's work on the PII protections) use something like Spacy to classify the tools into categories and then have the policy more dynamic. We could classify the tools based on what they can do (read,write,execute,network, ...) and with what (file, directory, repo, system, ...). Then have a policy that says "I don't want tools to ever touch network".

There might be overlap with MCP

[jhrozek]

@lukehinds @yrobla @ptelang WDYT? As I said above this was just and idea that came during code review, feel free to critique/close.

[yrobla]

I really think it's a good feature. With openinterpreter, i see it can lists contents inside your directory, open files, no matter what, and they are sent to the LLM. Is true that you need to authorize every action but will be good to have policies to prevent users from leaking certain contents or performing different types of actions: