feat: add Streamable HTTP transport support for remote MCP providers

[nabinchha]

📋 Summary

Adds streamable_http as a supported transport type for MCPProvider, resolving #357. Previously, remote MCP providers only supported SSE transport, causing silent 5-minute timeouts when connecting to Streamable HTTP-only endpoints (e.g. Tavily). Also adds DataDesigner.list_mcp_tool_names() for discovering available tools on a configured provider, and updates documentation across the board.

🔄 Changes

✨ Added

• streamable_http transport option on MCPProvider.provider_type (default remains "sse" for backwards compatibility)
• DataDesigner.list_mcp_tool_names() — convenience method to discover tools exposed by a configured MCP provider
• list_tool_names() helper in mcp.io module
• stub_streamable_http_provider test fixture
• Tests for streamable_http config validation, session creation, and CLI form builder

🔧 Changed

• MCPIOService._get_or_create_session() — routes streamable_http providers through streamablehttp_client, handles variable-length context manager results (3-tuple from streamablehttp_client vs 2-tuple from sse_client)
• MCPProviderFormBuilder — renamed _run_sse_form → _run_remote_form to handle both SSE and Streamable HTTP with appropriate labels
• MCPProviderController._select_provider — shows "Streamable HTTP" vs "SSE" in provider display

📚 Docs

• mcp-providers.md — updated overview table, fields table, Python examples, and YAML examples for both transports
• configure-mcp-cli.md — added Streamable HTTP to provider type selection and updated endpoint examples
• code_reference/mcp.md — updated MCPProvider description to mention both transports

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

• io.py#L222-L223 — Context manager result unpacking changed from read, write = ... to ctx_result[0], ctx_result[1] to handle both 2-tuple (SSE) and 3-tuple (Streamable HTTP) returns from the MCP SDK
• mcp.py#L47 — provider_type expanded from Literal["sse"] to Literal["sse", "streamable_http"] — this changes the Pydantic discriminator union; verify deserialization of existing configs still works

---

🤖 Generated with AI

[greptile-apps]

Greptile Summary

This PR extends MCPProvider with a streamable_http transport option alongside the existing "sse" default, resolving silent 5-minute timeouts when connecting to Streamable HTTP-only endpoints (e.g. Tavily). It also adds a DataDesigner.list_mcp_tool_names() convenience method for tool discovery, updates the CLI form builder to handle both remote transport types, and updates documentation throughout.

• mcp.py: provider_type widened from Literal["sse"] to Literal["sse", "streamable_http"]; default remains "sse", preserving backwards compatibility with existing Pydantic discriminated-union deserialization.
• io.py: New elif branch routes streamable_http providers through streamablehttp_client; context-manager result is now unpacked via index access (ctx_result[0], ctx_result[1]) to safely handle both the 2-tuple (SSE) and 3-tuple (Streamable HTTP) SDK returns.
• io.py / data_designer.py: New list_tool_names internal helper and DataDesigner.list_mcp_tool_names() public API with a minor signature inconsistency — list_tool_names requires timeout_sec: float while the sibling list_tools accepts float | None = None.
• CLI: _run_sse_form renamed to _run_remote_form with a provider_type parameter; provider selection list and display labels updated accordingly.
• Tests: 3-tuple mock correctly reflects the real streamablehttp_client SDK return type; good coverage across config validation, session creation, and CLI form builder.

Confidence Score: 4/5

• This PR is safe to merge; changes are well-scoped, backwards compatible, and covered by tests. The core transport routing and 3-tuple unpacking logic is correct.
• The PR correctly extends MCPProvider with streamable HTTP support while maintaining backwards compatibility. The Pydantic discriminated union continues to work because the default remains "sse". Session creation and context-manager unpacking safely handle both 2-tuple (SSE) and 3-tuple (Streamable HTTP) returns. Tests are thorough and the new mocks accurately reflect the MCP SDK behavior. The only notable issue is a minor API inconsistency in the list_tool_names signature (float instead of float | None), which does not affect runtime behaviour given how it is called today but limits future flexibility and creates type-level inconsistency with the sibling list_tools function.
• packages/data-designer-engine/src/data_designer/engine/mcp/io.py — list_tool_names signature inconsistency with list_tools.

Sequence Diagram

sequenceDiagram
    participant User
    participant DataDesigner
    participant MCPIOService
    participant sse_client
    participant streamablehttp_client
    participant stdio_client

    User->>DataDesigner: list_mcp_tool_names(name, timeout_sec)
    DataDesigner->>DataDesigner: lookup provider by name
    DataDesigner->>MCPIOService: list_tool_names(provider, timeout_sec)
    MCPIOService->>MCPIOService: _get_or_create_session(provider)

    alt provider_type == "stdio"
        MCPIOService->>stdio_client: stdio_client(params)
        stdio_client-->>MCPIOService: (read, write)
    else provider_type == "streamable_http"
        MCPIOService->>streamablehttp_client: streamablehttp_client(endpoint, headers)
        streamablehttp_client-->>MCPIOService: (read, write, get_session_id)
    else provider_type == "sse"
        MCPIOService->>sse_client: sse_client(endpoint, headers)
        sse_client-->>MCPIOService: (read, write)
    end

    MCPIOService->>MCPIOService: ctx_result[0], ctx_result[1] → read, write
    MCPIOService->>MCPIOService: ClientSession(read, write).initialize()
    MCPIOService->>MCPIOService: session.list_tools()
    MCPIOService-->>DataDesigner: [tool_name, ...]
    DataDesigner-->>User: list[str]

Loading

_{Last reviewed commit: 651ea68}

[greptile-apps]

_{9 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

packages/data-designer/src/data_designer/cli/forms/mcp_provider_builder.py
Update docstring to mention Streamable HTTP support

    """Builds interactive forms for MCP provider configuration.

    Supports both MCPProvider (remote SSE and Streamable HTTP) and LocalStdioMCPProvider (subprocess).
    """

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/data-designer/src/data_designer/cli/forms/mcp_provider_builder.py
Line: 22-24

Comment:
Update docstring to mention Streamable HTTP support

```suggestion
    """Builds interactive forms for MCP provider configuration.

    Supports both MCPProvider (remote SSE and Streamable HTTP) and LocalStdioMCPProvider (subprocess).
    """
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

[eric-tramel]

LGTM. Food for thought -- an Option C is to use a auto-detect/fallback on health check. This would mean that, unless it was specifically set by the user, we could "just" find which one of these two modes works and then run with it accordingly.

[eric-tramel]

Nit: Do we opt for StrEnum?

[nabinchha]

Nit: Do we opt for StrEnum?

We use a mix of string literals and string enums in this code base. We can definitely come back and change this to a StrEnum later.

[nabinchha]

LGTM. Food for thought -- an Option C is to use a auto-detect/fallback on health check. This would mean that, unless it was specifically set by the user, we could "just" find which one of these two modes works and then run with it accordingly.

True, but we'd have to update the state from health check which we currently don't do. Option C presented in the gh issue was to auto fall back in a try-catch, but that adds a bit more latency. We can re-visit this later if we hear more feedback.