Skip to content

feat(typescript): MCP tool-UI passthrough in MCPToolProvider - #597

Merged
cornelcroi merged 3 commits into
mainfrom
feat/typescript-mcp-tool-ui
Jul 14, 2026
Merged

feat(typescript): MCP tool-UI passthrough in MCPToolProvider#597
cornelcroi merged 3 commits into
mainfrom
feat/typescript-mcp-tool-ui

Conversation

@cornelcroi

Copy link
Copy Markdown
Collaborator

Issue Link

Closes #596

Summary

Makes the TypeScript MCPToolProvider carry a server-advertised widget through to the caller instead of flattening the MCP result to text — so a TS GroundedAgent can render the widgets of the same MCP server that backs a ChatGPT App. Mirrors the Swift and Python (#595) providers.

Stacked on #593 (base feat/typescript-grounded-agent-tool-ui, which adds TS ToolResult/UIPayload). Merge #593 first; GitHub retargets this to main.

Changes

  • Parse each tool's _meta at connect time: UI resource URI (_meta.ui.resourceUri or the OpenAI openai/outputTemplate alias) and visibility. App-only tools (visibility excludes model) are filtered out of this.tools — never advertised — while staying in toolClientMap.
  • Each MCP tool's func is now real: it calls the server and returns a ToolResult. callMCPTool reads structuredContent, and when the tool advertised a UI it fetches the resource (readResource, cached per client) and builds a UIPayload.
  • toolHandler invokes func with the full input and routes only the text to the model.
  • Also fixes a latent bug: GroundedAgent captures tool results via the tool's func; the previous dummy func + bespoke toolHandler override bypassed it, so MCP tool calls weren't captured at all. Now they are.

User experience

Point a GroundedAgent's gatherer at an MCP server whose tool advertises a widget; consume the agent's stream and the caller gets the widget (a { ui } chunk) before the grounded text.

Checklist

  • Backward compatible — a server with no _meta.ui sends the same model-facing text (success / isError / exception / unknown-tool strings all match the old override); full input preserved (no messages-key drop, no throw on undefined)
  • Lightweight / additive — no new dependency (@modelcontextprotocol/sdk stays an optional peer)
  • Tests added (widget passthrough, alias, blob, fetch-failure, cache, visibility, messages-key + undefined input); npm run build/lint/coverage green (169 tests, 16 suites)
  • Docs updated (MCP tool-provider page Tool UI section)
  • Reviewed by the TypeScript expert; the one REQUIRED item (full-input handling vs base processTool) fixed

Note: touches the same docs section as #595 (Python MCP) — resolve any overlap by keeping this three-language wording.

Code written by Claude (Opus 4.8), architected and approved by @cornelcroi.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c436a6b73b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread typescript/src/tools/mcpToolProvider.ts Outdated
Comment on lines +333 to +334
const tool = this.tools.find((t) => t.name === toolName);
const result = tool ? await tool.func(inputData) : await this.callMCPTool(toolName, inputData);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject model calls to app-only MCP tools

When a tool has _meta.ui.visibility that excludes model, it is removed from this.tools, but this fallback still executes it if a tool-use block names it. In any environment where the model/provider can emit or replay an arbitrary tool name, an app-only MCP tool becomes invocable through the model tool loop instead of being rejected; keeping such tools callable for the UI needs a separate path that is not toolHandler.

Useful? React with 👍 / 👎.

Surface widgets advertised by an MCP server instead of flattening results to
text. Each tool's _meta is parsed at connect time (ui resource URI via
_meta.ui.resourceUri or the openai/outputTemplate alias, plus visibility);
app-only tools are kept callable but filtered out of this.tools. Each MCP
tool's func is now real: it calls the server and returns a ToolResult
(joined text -> content, structuredContent -> structuredContent, and a
UIPayload fetched via readResource + cached per client when the tool advertised
a UI). toolHandler invokes func with the full input and routes only the text to
the model.

This also fixes a latent gap: GroundedAgent captures via the tool's func, which
the previous dummy-func + bespoke toolHandler override bypassed, so MCP tool
calls were never captured. Additive: a server advertising no _meta.ui sends the
same text to the model as before. Mirrors the Swift and Python providers.

Closes #596

Code written by Claude (Opus 4.8), architected and approved by @cornelcroi.
Drives an MCP tool through the provider's toolHandler inside a GroundedAgent and
asserts the presenter runs on the captured facts. Before the func/toolHandler
rework this capture never happened (grounding silently degraded to the
gatherer's draft), so this locks the fix in.

Code written by Claude (Opus 4.8), architected and approved by @cornelcroi.
toolHandler now only executes tools present in this.tools (model-visible). A
tool-use block naming an app-only or unknown tool returns a not-found result
instead of being executed, so an app-only MCP tool can never be invoked through
the model loop by an emitted/replayed name. App-only tools stay in
toolClientMap for a future host-side call path.

Code written by Claude (Opus 4.8), architected and approved by @cornelcroi.
@cornelcroi
cornelcroi force-pushed the feat/typescript-mcp-tool-ui branch from bd3668a to 77677c6 Compare July 14, 2026 20:28
@cornelcroi
cornelcroi changed the base branch from feat/typescript-grounded-agent-tool-ui to main July 14, 2026 20:28
@cornelcroi
cornelcroi merged commit d0c3c01 into main Jul 14, 2026
6 of 7 checks passed
@cornelcroi
cornelcroi deleted the feat/typescript-mcp-tool-ui branch July 14, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TypeScript: MCP tool-UI passthrough (widgets from MCP servers) in MCPToolProvider

1 participant