Skip to content

feat(mcp): per-tool options and progress update for MCPToolset#5694

Merged
longcw merged 8 commits into
mainfrom
longc/mcp-toolset-async-mode
Jul 17, 2026
Merged

feat(mcp): per-tool options and progress update for MCPToolset#5694
longcw merged 8 commits into
mainfrom
longc/mcp-toolset-async-mode

Conversation

@longcw

@longcw longcw commented May 9, 2026

Copy link
Copy Markdown
Contributor

Summary

MCPToolset now extends AsyncToolset. Per-tool behavior is configured with MCPToolOptions, keyed by tool name via tool_options; any tool not listed there is a plain blocking call.

MCPToolOptions keeps the three concerns independent:

  • flags: ToolFlag — e.g. CANCELLABLE, IGNORE_ON_ENTER (mirrors @function_tool).
  • on_duplicate: DuplicateMode — duplicate-call policy (mirrors @function_tool).
  • forward_progress: bool — forward the MCP server's report_progress notifications to ctx.update(), so the tool runs in the background and the agent can narrate progress.

When any tool is ToolFlag.CANCELLABLE, lk_agents_get_running_tasks / lk_agents_cancel_task are auto-exposed to the LLM by the activity.

AgentSession(tools=[
    MCPToolset(
        id="mcp",
        mcp_server=MCPServerHTTP(url="...", client_session_timeout_seconds=120),
        tool_options={
            "book_flight": MCPToolOptions(
                flags=ToolFlag.CANCELLABLE,
                on_duplicate="confirm",
                forward_progress=True,
            ),
        },
    ),
])

Example

examples/voice_agents/mcp/server.py ships a fast get_weather (sync, no progress) and a long-running book_flight that emits ctx.report_progress updates over ~70s. mcp-agent.py configures only book_flight (cancellable + forward progress) and bumps client_session_timeout_seconds=120 since the per-request timeout otherwise fires before it returns.

MCPToolset now extends AsyncToolset and gains an `async_mode=True` flag.
When set, MCP tools are built with an `AsyncRunContext` parameter that
forwards MCP progress notifications via `ctx.update`, so the agent can
narrate progress while a slow MCP tool runs in the background instead of
blocking the reply loop.

`AsyncToolset` no longer stores `get_running_tasks` / `cancel_task` in
`_tools`; they're exposed through a `tools` property override that
subclasses can condition. `MCPToolset` overrides it to expose the
helpers only when `async_mode=True`, so the legacy MCP UX is unchanged
when the flag is off.

`MCPServer.list_tools` now caches per `async_mode`, and the example MCP
server gains a long-running `book_flight` tool that emits progress
updates.
@chenghao-mou
chenghao-mou requested a review from a team May 9, 2026 01:59
devin-ai-integration[bot]

This comment was marked as resolved.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@theomonnom theomonnom left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, let's have async_mode enabled by default?

maybe the arguments becomes something like nonblocking_tools: list[str]?

Public API on MCPToolset becomes `nonblocking_tools: bool | list[str]`
defaulting to `True`, so MCP tools run non-blocking by default and callers
can opt specific tools in or out by name. Empty list / `False` falls back
to the legacy blocking behavior.

MCPServer.list_tools routes the per-tool decision down to
_make_function_tool, which still takes a bool, and caches the resolved
bool|frozenset key.
@longcw
longcw requested a review from theomonnom May 13, 2026 11:01
@longcw longcw changed the title feat(mcp): add async_mode to MCPToolset feat(mcp): add nonblocking_tools to MCPToolset May 13, 2026
@theomonnom

Copy link
Copy Markdown
Member

Let's wait for #5711

devin-ai-integration[bot]

This comment was marked as resolved.

longcw added 2 commits July 13, 2026 16:23
Re-implement MCP async support on the reworked AsyncToolset API: MCPToolset extends AsyncToolset, non-blocking MCP tools carry ToolFlag.CANCELLABLE and forward progress through ctx.update(), and get_running_tasks/cancel_task are auto-exposed by the activity when a cancellable tool is present rather than added by the toolset.
Each MCP tool is configured via MCPToolOptions(flags, on_duplicate, forward_progress) keyed by name through MCPToolset(tool_options=...); tools not listed there are plain blocking calls. flags/on_duplicate mirror @function_tool and forward_progress forwards report_progress to ctx.update(), decoupling the non-blocking, cancellable, and duplicate-handling concerns that the single nonblocking_tools flag conflated. MCPServer now caches only the raw tool descriptors and rebuilds tools per resolved options.
@longcw longcw changed the title feat(mcp): add nonblocking_tools to MCPToolset feat(mcp): per-tool options for MCPToolset Jul 13, 2026
list_tools takes the tool_options mapping directly instead of a resolver callable — MCPToolset was the only caller and just forwarded dict.get. Also tighten the MCPToolOptions/MCPToolset docstrings and comments.
@longcw longcw changed the title feat(mcp): per-tool options for MCPToolset feat(mcp): per-tool options and progress update for MCPToolset Jul 13, 2026
Comment thread livekit-agents/livekit/agents/llm/mcp.py Outdated
Comment thread livekit-agents/livekit/agents/llm/mcp.py Outdated
Rename forward_progress to report_progress and resolve per-tool options against a full default map instead of inline getattr defaults.
@longcw
longcw merged commit 77b5d8b into main Jul 17, 2026
22 of 23 checks passed
@longcw
longcw deleted the longc/mcp-toolset-async-mode branch July 17, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants