[responsesAPI][7] Browser, Container MCP tools for non harmony models

[qandrew]

Purpose

• add browser tool, container tool to ParsableContext
• note, we don't need env var VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS for ResponsesParser because that's specific to GPTOSS
• for the future, think about how to manage MCP list better

Test Plan

minimax server

VLLM_USE_EXPERIMENTAL_PARSER_CONTEXT=1 vllm serve MiniMaxAI/MiniMax-M2   --tensor-parallel-size 4   --tool-call-parser minimax_m2   --reasoning-parser minimax_m2    --enable-auto-tool-choice --trust-remote-code  --tool-server=localhost:8081/container,localhost:8081/browser,localhost:8081/python

gptoss server

VLLM_GPT_OSS_SYSTEM_TOOL_MCP_LABELS=web_search_preview,container,code_interpreter CUDA_VISIBLE_DEVICES=2,3 with-proxy vllm serve "/data/users/axia/checkpoints/gpt-oss-120b" -tp 2     --trust-remote-code  --tool-server=localhost:8081/container,localhost:8081/browser,localhost:8081/python

container tool

curl -X POST "http://localhost:8000/v1/responses"   -H "Content-Type: application/json"   -H "Authorization: Bearer dummy-api-key"   -d '{
        "model": "MiniMaxAI/MiniMax-M2",
        "input": "Use the container tool to run `ls` and return the output.",
        "tools": [
          {
            "type": "mcp",
            "server_label": "container",
            "headers": {"test": "test"},
            "server_url": "IGNORED"
          }
        ]
      }'

{
    "id": "resp_9a842ebddec3f652",
    "created_at": 1764892256,
    "incomplete_details": null,
    "instructions": null,
    "metadata": null,
    "model": "MiniMaxAI/MiniMax-M2",
    "object": "response",
    "output": [
        {
            "id": "rs_a25116aeb3e78bb8",
            "summary": [],
            "type": "reasoning",
            "content": [
                {
                    "text": "Okay, the user wants me to use the \"container\" tool to execute the `ls` command. Let me check the available tools again. The tool is named \"container\" and has the type \"mcp\". But looking at the parameters, the allowed_tools and other fields are empty. Hmm, that seems odd. Maybe the tool's parameters aren't properly defined here?\n\nWait, the tool's description says \"Server URL: IGNORED\", which might mean that the actual server isn't configured. But the user specifically asked to use the container tool. Perhaps I need to call it with the name \"container\" as specified. Even though the parameters are empty, maybe I should just pass an empty arguments object. Let me try that.\n\nSo, I'll structure the tool call with the name \"container\" and an empty arguments JSON. The response should be whatever the tool returns, but since the allowed_tools are null, there might not be any output. Alternatively, the tool might not be set up correctly. But I have to follow the user's request as given.\n\nI should respond by making the call and returning the output, even if it's empty. If the tool fails, maybe an error will occur, but the user might want to see the attempt. Alternatively, if the tool isn't properly configured, the system might return an error. But since the user asked to use the container tool, I'll proceed with the call as instructed.\n",
                    "type": "reasoning_text"
                }
            ],
            "encrypted_content": null,
            "status": null
        },
        {
            "id": "mcp_9e9f0d2a6916b8f6",
            "arguments": "{}",
            "name": "container",
            "server_label": "container",
            "type": "mcp_call",
            "error": null,
            "output": "Error executing tool exec: 1 validation error for execArguments\ncmd\n  Field required [type=missing, input_value={}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.11/v/missing",
            "status": "completed"
        },
        {
            "id": "rs_9a344edddbaa222c",
            "summary": [],
            "type": "reasoning",
            "content": [
                {
                    "text": "Okay, the user asked me to use the container tool to run `ls` and return the output. Let me think about how to approach this.\n\nFirst, I need to check what tools are available. According to the tools section, there's a \"container\" tool with the type \"mcp\". However, looking at the parameters, it seems like the allowed_tools and other fields are empty. That might indicate that the tool's parameters aren't properly specified here. But the user specifically mentioned using the \"container\" tool, so I should attempt to use it.\n\nThe initial tool call I made didn't include any arguments because the parameters section was empty. However, the tool returned an error saying that the `cmd` field is missing and is required. This suggests that the tool expects a command to be passed, even though it wasn't specified in the tool's parameters here.\n\nSo, I need to correct the tool call to include the `cmd` parameter. The command the user wants to run is `ls`, which lists files and directories. I should structure the tool call with `cmd` set to `[\"ls\"]` as an array. The error message indicates that the cmd field must be provided, and it expects a list of strings (the command and its arguments).\n\nThe next step is to re-execute the tool call with the correct parameters. The user might be expecting the output of the `ls` command, which could be a list of files in the current directory. If the tool is properly configured, it should return that output. If not, another error might occur, but I have to follow the user's request to use the container tool with the `ls` command.\n\nSo, the corrected tool call should have `cmd` as a list containing `ls`. This should execute the `ls` command in the container and return the output, which I can then present to the user.\n",
                    "type": "reasoning_text"
                }
            ],
            "encrypted_content": null,
            "status": null
        },
        {
            "id": "mcp_95123a91931680d0",
            "arguments": "{\"cmd\": \"[\\\"ls\\\"]\"}",
            "name": "container",
            "server_label": "container",
            "type": "mcp_call",
            "error": null,
            "output": "\nstdout:\nbin\nboot\ndev\netc\nhome\nkotlin-compiler-2.2.10.zip\nlib\nlib32\nlib64\nlibx32\nmedia\nmnt\nmsl_rl_requirements.txt\nopt\nproc\nroot\nrun\nsbin\nsrv\nsys\ntmp\nusr\nvar\nworkspace\n",
            "status": "completed"
        },
        {
            "id": "rs_94adcbf98cbaaf31",
            "summary": [],
            "type": "reasoning",
            "content": [
                {
                    "text": "Okay, let me think through what's happening here. The user asked me to use the container tool to run the `ls` command and return the output. Initially, I tried to call the container tool, but I received an error message indicating that the `cmd` parameter was required but missing.\n\nThe error message specified: \"Field required [type=missing, input_value={}, input_type=dict]\" and pointed out that the `cmd` field was missing. This makes sense because when executing commands in a container environment, I need to specify the command and its arguments.\n\nLooking back at the tool description, I noticed that although the tool's parameters were mostly empty, it seems that the correct approach was to include the `cmd` parameter as a list of strings. This is typical for command-line tools where the first element is the command itself and subsequent elements are arguments.\n\nSo I corrected my approach by calling the container tool again, but this time with the `cmd` parameter set to `[\"ls\"]`. This tells the container to execute the `ls` command without any additional arguments, which should simply list the files and directories in the current working directory.\n\nThe tool executed successfully this time and returned a list of directories and files in the container. The output includes common Linux directories like `bin`, `boot`, `dev`, `etc`, `home`, etc., as well as some specific files like `kotlin-compiler-2.2.10.zip` and `msl_rl_requirements.txt`. This output appears to be the complete directory listing of the root directory in the container, which is exactly what the user requested.\n\nI should now present this output to the user in a clear, readable format. Since the user specifically asked for the output of the `ls` command, I'll just display it as is without any additional formatting or commentary, since that seems to be what they're looking for.\n",
                    "type": "reasoning_text"
                }
            ],
            "encrypted_content": null,
            "status": null
        },
        {
            "id": "msg_9c46daf99374800d",
            "content": [
                {
                    "annotations": [],
                    "text": "\n\nThe output of running `ls` in the container is:\n\n```\nbin\nboot\ndev\netc\nhome\nkotlin-compiler-2.2.10.zip\nlib\nlib32\nlib64\nlibx32\nmedia\nmnt\nmsl_rl_requirements.txt\nopt\nproc\nroot\nrun\nsbin\nsrv\nsys\ntmp\nusr\nvar\nworkspace\n```",
                    "type": "output_text",
                    "logprobs": null
                }
            ],
            "role": "assistant",
            "status": "completed",
            "type": "message"
        }
    ],
    "parallel_tool_calls": true,
    "temperature": 1.0,
    "tool_choice": "auto",
    "tools": [
        {
            "server_label": "container",
            "type": "mcp",
            "allowed_tools": null,
            "authorization": null,
            "connector_id": null,
            "headers": {
                "test": "test"
            },
            "require_approval": null,
            "server_description": null,
            "server_url": "IGNORED"
        }
    ],
    "top_p": 0.95,
    "background": false,
    "max_output_tokens": 195536,
    "max_tool_calls": null,
    "previous_response_id": null,
    "prompt": null,
    "reasoning": null,
    "service_tier": "auto",
    "status": "completed",
    "text": null,
    "top_logprobs": null,
    "truncation": "disabled",
    "usage": {
        "input_tokens": 1072,
        "input_tokens_details": {
            "cached_tokens": 944,
            "input_tokens_per_turn": [],
            "cached_tokens_per_turn": []
        },
        "output_tokens": 1175,
        "output_tokens_details": {
            "reasoning_tokens": 0,
            "tool_output_tokens": 0,
            "output_tokens_per_turn": [],
            "tool_output_tokens_per_turn": []
        },
        "total_tokens": 2247
    },
    "user": null,
    "input_messages": null,
    "output_messages": null
}

browser tool

curl -X POST "http://localhost:8000/v1/responses"   -H "Content-Type: application/json"   -H "Authorization: Bearer dummy-api-key"   -d '{
        "model": "MiniMaxAI/MiniMax-M2",
        "input": "Whats the latest news in New York City?",
        "tools": [
          {
            "type": "mcp",
            "server_label": "web_search_preview",
            "headers": {"test": "test"},
            "server_url": "IGNORED"
          }
        ]
      }'

{
    "id": "resp_b60b76e1063c52ce",
    "created_at": 1764789823,
    "incomplete_details": null,
    "instructions": null,
    "metadata": null,
    "model": "MiniMaxAI/MiniMax-M2",
    "object": "response",
    "output": [
        {
            "id": "rs_9c054200c951ade0",
            "summary": [],
            "type": "reasoning",
            "content": [
                {
                    "text": "Okay, the user is asking for the latest news in New York City. This is a straightforward request for current information that I'll need to search for online.\n\nFirst, I need to determine how to get this information. Looking at the tools available to me, I have access to a web_search_preview tool. This seems perfect for getting current news information. The tool description mentions it can search the web, which is exactly what I need to find the latest news about New York City.\n\nNow I need to formulate an appropriate search query. The user wants news specifically about New York City, so I should make sure my search query is focused on that location. A simple and effective query would be \"latest news New York City\" - this is direct, includes the location, and specifies that I want recent news (using \"latest\").\n\nI could potentially make the query more specific, like \"New York City breaking news today\" or \"New York City current events,\" but that might be too restrictive. The basic query \"latest news New York City\" should return a good range of current news stories from various sources about the city.\n\nI don't need to add any other parameters to my search. The tool seems to only require the search query itself. I don't need to specify a date range or news source since the user wants general latest news.\n\nSo I'll call the web_search_preview tool with the query \"latest news New York City\" to get up-to-date information about what's happening in New York City. This should provide the user with the current news they're looking for.\n",
                    "type": "reasoning_text"
                }
            ],
            "encrypted_content": null,
            "status": null
        },
        {
            "id": "mcp_816d39baa6c5e093",
            "arguments": "{\"query\": \"latest news New York City\"}",
            "name": "web_search_preview",
            "server_label": "web_search_preview",
            "type": "mcp_call",
            "approval_request_id": null,
            "error": null,
            "output": "[0] latest news New York City\n**viewing lines [0 - 5] of 5**\n\nL0: # Search Results\nL1: \nL2:   * 【0†Google News - New York - Latest】 Read full articles, watch videos, browse\nL3:  thousands of titles and more on the \"New York\" topic with Google News.\nL4:   * 【1†Google News - New York - Latest】 Read full articles, watch videos, browse\nL5:  thousands of titles and more on the \"New York\" topic with Google News.",
            "status": "completed"
        },
        {
            "id": "rs_ad406ba4645a849f",
            "summary": [],
            "type": "reasoning",
            "content": [
                {
                    "text": "Okay, I've received the results from my search for the latest news in New York City. Let me analyze what I got back.\n\nThe search returned 5 lines of results, with the content being quite minimal. It seems the search results only included two entries, both of which appear to be \"Google News - New York - Latest\" links. These are essentially the same result listed twice, with a brief description about being able to read full articles, watch videos, and browse thousands of titles on the \"New York\" topic.\n\nThis is not very helpful for providing the user with actual news content. The search results are too generic and don't provide any specific news stories or headlines about New York City. They're just links to Google News' New York section.\n\nLooking back at how I structured my search query, I used \"latest news New York City\" which should have been specific enough. However, the web_search_preview tool might be returning general results rather than specific news articles. The tool might be limited in its ability to provide detailed search results.\n\nGiven these limitations, I need to acknowledge to the user that I couldn't provide them with the specific news they were looking for. I should be straightforward about this and suggest alternative ways they might find the information they need.\n\nThe best approach is to:\n1. Admit that I couldn't retrieve specific news stories\n2. Provide links to reliable news sources (Google News, BBC, and CNN) that cover New York City news\n3. Make it clear that by visiting these sources, they can find the latest news about New York City\n\nThis way, even though I couldn't directly answer their question with specific news stories, I'm still being helpful by pointing them in the right direction to get the information they're seeking.\n",
                    "type": "reasoning_text"
                }
            ],
            "encrypted_content": null,
            "status": null
        },
        {
            "id": "msg_b8a807c96ac7c364",
            "content": [
                {
                    "annotations": [],
                    "text": "\n\nIt seems I couldn't find specific news stories for New York City. However, you can check Google News for the latest updates at \"Google News - New York - Latest\" or other reliable news sources like BBC or CNN for information about New York City.",
                    "type": "output_text",
                    "logprobs": null
                }
            ],
            "role": "assistant",
            "status": "completed",
            "type": "message"
        }
    ],
    "parallel_tool_calls": true,
    "temperature": 1.0,
    "tool_choice": "auto",
    "tools": [
        {
            "server_label": "web_search_preview",
            "type": "mcp",
            "allowed_tools": null,
            "authorization": null,
            "connector_id": null,
            "headers": {
                "test": "test"
            },
            "require_approval": null,
            "server_description": null,
            "server_url": "IGNORED"
        }
    ],
    "top_p": 0.95,
    "background": false,
    "max_output_tokens": 195925,
    "max_tool_calls": null,
    "previous_response_id": null,
    "prompt": null,
    "reasoning": null,
    "service_tier": "auto",
    "status": "completed",
    "text": null,
    "top_logprobs": null,
    "truncation": "disabled",
    "usage": {
        "input_tokens": 683,
        "input_tokens_details": {
            "cached_tokens": 496,
            "input_tokens_per_turn": [],
            "cached_tokens_per_turn": []
        },
        "output_tokens": 745,
        "output_tokens_details": {
            "reasoning_tokens": 0,
            "tool_output_tokens": 0,
            "output_tokens_per_turn": [],
            "tool_output_tokens_per_turn": []
        },
        "total_tokens": 1428
    },
    "user": null,
    "input_messages": null,
    "output_messages": null
}

[mergify]

Documentation preview: https://vllm--29989.org.readthedocs.build/en/29989/

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Reuse tool call_id when emitting tool output

In ParsableContext.call_python_tool the tool output is created with a new random call_id instead of the last_msg.call_id from the assistant tool call, and the same pattern is used in the search and container helpers. The next turn builds a tool message whose tool_call_id comes from this new ID, so it no longer matches the assistant tool call that triggered it, producing an invalid transcript and preventing the model from associating the tool response with its request.

Useful? React with 👍 / 👎.

[qandrew]

i think right now we are ad-hoc but it's fine, because we only support 3 MCP tools. When we support generic MCP tools (for both GPTOSS and non GPTOSS) we could have better logic here

[yeqcharlotte]

could you leave a TODO here assigned to yourself