💡 MCP Apps Integration: Interactive Tool UIs in Avatar-Guided Overlays

[github-actions[bot]]

Summary

Implement TalkTerm as an MCP Apps host, rendering interactive server-provided HTML UIs (dashboards, forms, data visualizations) in the right-panel overlay system via sandboxed iframes. The avatar narrates and guides users through these interactive UIs conversationally, making complex tool outputs accessible to non-technical users.

Market Signal

MCP Apps (SEP-1865) launched January 2026 with the full spec release candidate in July 2026. Already supported by Claude Desktop, ChatGPT, VS Code Insiders, and Goose. 97 million monthly MCP SDK downloads and 5,800+ servers create a massive ecosystem of tools that could contribute interactive UIs to TalkTerm. Being an MCP Apps host is becoming table-stakes for AI desktop clients.

User Signal

TalkTerm's OutputPanel currently plans 6 fixed display modes (document, comparison table, clustered cards, etc.). MCP Apps would allow ANY MCP server to contribute rich, interactive UIs without TalkTerm needing to build each display mode. This transforms TalkTerm from a closed UI system into an extensible platform. Existing idea #66 (Conversational MCP Onboarding) covers MCP server discovery but not the MCP Apps UI rendering capability.

Technical Opportunity

The architecture already defines a right-panel OutputPanel (UX-DR4, 380px) with sandboxed display modes. Electron natively supports iframe sandboxing. The MCP Apps spec uses JSON-RPC over postMessage for bidirectional UI-host communication, which maps cleanly onto the existing IPC bridge pattern. The @modelcontextprotocol/ext-apps SDK provides the App class for UI-host communication. Avatar narration of MCP App UIs leverages the existing TTS pipeline and the avatar announcement pattern (UX-DR4: "Avatar announces content verbally before panel appears").

Assessment

Dimension	Score	Rationale
Feasibility	high	MCP Apps spec is stable with production SDK; Electron iframe sandboxing is native; maps directly onto existing OutputPanel architecture
Impact	high	Transforms TalkTerm from closed UI system to extensible platform; unlocks 5,800+ MCP server ecosystem for interactive UIs
Urgency	high	Spec RC shipping July 2026; Claude Desktop and ChatGPT already support it; first-mover advantage for avatar-guided MCP Apps UX

Adversarial Review

Strongest objection: Claude Desktop already supports MCP Apps. Why would users choose TalkTerm's implementation over the native Claude client?

Rebuttal: Claude Desktop renders MCP Apps in a flat chat UI. TalkTerm renders them in a dedicated overlay panel with avatar verbal guidance — the avatar can walk non-technical users through complex dashboards, point out key data, and offer to interact with the UI on their behalf. For TalkTerm's target persona (non-technical knowledge workers), guided visual interaction is the difference between "here is a dashboard" and "let me show you what this means." The avatar layer is the differentiator.

Suggested Next Step

Create a technical spike story evaluating MCP Apps host implementation: render a sandboxed iframe in OutputPanel, implement JSON-RPC postMessage bridge, and test with 2-3 existing MCP Apps servers (e.g., a data visualization server and a form server).