From 77030e8f5191694e8b0f0e1a6b6613de0298795a Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Tue, 20 May 2025 22:46:27 -0700 Subject: [PATCH 1/3] Adding info for remote servers --- docs-v2/pages/connect/mcp/developers.mdx | 200 ++++++++++++++++++++--- docs-v2/pages/connect/mcp/openai.mdx | 54 +----- 2 files changed, 187 insertions(+), 67 deletions(-) diff --git a/docs-v2/pages/connect/mcp/developers.mdx b/docs-v2/pages/connect/mcp/developers.mdx index 90c2782f92a13..a5ed0e7d36ad9 100644 --- a/docs-v2/pages/connect/mcp/developers.mdx +++ b/docs-v2/pages/connect/mcp/developers.mdx @@ -1,42 +1,184 @@ -import { Steps } from 'nextra/components' +import { Callout, Steps, Tabs } from 'nextra/components' -# Deploy Pipedream MCP to your agent -Deploy Pipedream's MCP servers to your application or agent and make authenticated request on behalf of your customers. Refer to our reference implementation [here](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md). +# Add Pipedream MCP to your app or agent -## Quickstart +Add Pipedream's MCP server to your application or agent to make tool calls on behalf of your users to {process.env.PUBLIC_APPS}+ APIs and 10,000+ tools. - +## Overview -#### Prerequisites +Pipedream's MCP server code is [publicly available in GitHub](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md), and you have two options for using Pipedream's MCP server in your app: -To run your own MCP server, you'll need: +1. [Use Pipedream's remote MCP server](#use-pipedreams-remote-mcp-server) +2. [Self-host Pipedream's MCP server](#self-host-pipedreams-mcp-server) + +### Pipedream concepts to understand + +The MCP server accepts two route params: + +**`external_user_id`** + +- This is your user’s ID, in your system: whatever you use to uniquely identify them +- Any requests made to that route are coupled to that end user and uses the connected account tied to that user +- [See here](/connect/api/#external-users) for more detail + +**`app`** + +- The app's "name slug" (the unique identifier for the app) +- [See below](#discover-available-mcp-servers) for discovering available apps + +### Account connections + +One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage. + +You can handle account connections in one of two ways in your app: + +### Add a button in your UI +- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI +- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com) + +### Return a link +- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account +- There is no implementation required for this option since it's already handled in Pipedream's MCP server +- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example: + +``` +https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app={appSlug} +``` + +## Getting started + +### Prerequisites + +For either option, you'll need: 1. A [Pipedream account](https://pipedream.com/auth/signup) 2. A [Pipedream project](/projects/#creating-projects). Accounts connected via MCP will be stored here. 3. [Pipedream OAuth credentials](/rest-api/auth/#oauth) -#### Set up environment variables +#### Set up your environment Set the following environment variables: ```bash PIPEDREAM_CLIENT_ID=your_client_id PIPEDREAM_CLIENT_SECRET=your_client_secret -PIPEDREAM_PROJECT_ID=your_project_id -PIPEDREAM_ENVIRONMENT=development +PIPEDREAM_PROJECT_ID=your_project_id # proj_xxxxxxx +PIPEDREAM_ENVIRONMENT=development # development | production +``` + +### Discover available MCP servers + +Pipedream provides [{process.env.PUBLIC_APPS}+ APIs as MCP servers](https://mcp.pipedream.com). Each server corresponds to an app integration (like Notion, Gmail, or Slack) and has its own specific set of tools that you can expose to OpenAI. + + + +```javascript +// Get all available apps (paginated) +const apps = await pd.getApps(); + +// Each app has these key properties: +// - name_slug: Used in the MCP server URL (e.g., "notion", "gmail", "slack") +// - name: Display name (e.g., "Notion", "Gmail", "Slack") +``` + + +```javascript +// Search by app name +const notionApps = await pd.getApps({ q: "notion" }); +const gmailApps = await pd.getApps({ q: "gmail" }); +const slackApps = await pd.getApps({ q: "slack" }); +``` + + + +### Use Pipedream's remote MCP server + + +Pipedream MCP server supports both SSE and streamable HTTP transport types, with no configuration required by the developer or MCP client. + + +#### Base URL + +``` +https://mcp.pipedream.net +``` + +#### Authentication + +To authenticate requests to Pipedream's MCP server, you need to include an access token with every HTTP request. Here's how to get it: + + + +```javascript +import { createBackendClient } from "@pipedream/sdk/server"; + +// Initialize the Pipedream client with the SDK +const pd = createBackendClient({ + environment: PIPEDREAM_ENVIRONMENT, + credentials: { + clientId: PIPEDREAM_CLIENT_ID, + clientSecret: PIPEDREAM_CLIENT_SECRET, + }, + projectId: PIPEDREAM_PROJECT_ID +}); + +// Get access token for MCP server auth +const accessToken = await pd.rawAccessToken(); + +console.log(accessToken); +``` + + +```bash +curl -s -X POST https://api.pipedream.com/v1/oauth/token \ + -H "Content-Type: application/json" \ + -d '{ + "grant_type": "client_credentials", + "client_id": "'$PIPEDREAM_CLIENT_ID'", + "client_secret": "'$PIPEDREAM_CLIENT_SECRET'" + }' +``` + + + +#### Required headers + +Include these headers in every HTTP request: + +```javascript +{ + Authorization: `Bearer ${PIPEDREAM_ACCESS_TOKEN}`, + "x-pd-project-id": PIPEDREAM_PROJECT_ID, // proj_xxxxxxx + "x-pd-environment": PIPEDREAM_ENVIRONMENT // development | production +} +``` + +### Self-host Pipedream's MCP server + +#### Using the `Dockerfile` + +If you have Docker installed locally, you can build and run the container from the [reference implementation](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/Dockerfile): + +```console +> docker build -t pipedream-connect . +> docker run -d --name pd-mcp -p 3010:3010 --env-file .env pipedream-connect:latest ``` -#### Run the MCP server +This exposes a generic MCP server at [http://localhost:3010/:external_user_id/:app](http://localhost:3010/:external_user_id/:app). + +#### Start the server with Streamable HTTP Transport -Pipedream's MCP servers can run in two modes: +```bash +pnpm dev:http +``` -##### Stdio (for local testing) +You can use the optional env var `PD_SDK_DEBUG` to print out all the requests and responses going to the Connect API: ```bash -npx @pipedream/mcp stdio --app slack --external-user-id user123 +PD_SDK_DEBUG=true pnpm dev:http ``` -##### SSE (for hosting) +#### Start the server with SSE ```bash npx @pipedream/mcp sse @@ -46,13 +188,9 @@ This exposes routes at: - `GET /:external_user_id/:app` - App-specific SSE connection endpoint - `POST /:external_user_id/:app/messages` - App-specific message handler -Where: -- `:external_user_id` is your user's unique identifier in your system -- `:app` is the app slug (e.g., "slack", "github") - #### Customize for your needs -You can host and customize the MCP server for your specific requirements: +You can also host and customize the MCP server for your specific requirements: ```bash # Clone the repo @@ -71,4 +209,24 @@ npm run start:sse:prod See the [MCP server README](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md) for detailed instructions on customization options. - \ No newline at end of file +### Use the MCP inspector + +The [MCP inspector](https://modelcontextprotocol.io/docs/tools/inspector) can be helpful when debugging tool calls. + +```bash +npx @modelcontextprotocol/inspector +``` + +Enter the server URL: + +If using Pipedream's remote server: + +``` +https://mcp.pipedream.net/{external_user_id}/{app_slug} +``` + +If running locally: + +``` +http://localhost:3010/{external_user_id}/{app_slug} +``` diff --git a/docs-v2/pages/connect/mcp/openai.mdx b/docs-v2/pages/connect/mcp/openai.mdx index c6399d8fc1a72..390fbf1269970 100644 --- a/docs-v2/pages/connect/mcp/openai.mdx +++ b/docs-v2/pages/connect/mcp/openai.mdx @@ -5,10 +5,10 @@ import { Callout, Tabs, Steps } from 'nextra/components' Access {process.env.PUBLIC_APPS}+ APIs and 10,000+ tools in OpenAI using Pipedream Connect. MCP makes it easy to extend the capabilities of any LLM or agent, and Pipedream offers drop-in support for [calling tools in OpenAI](https://platform.openai.com/docs/guides/tools?api-mode=responses) . -Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more below](#account-connection). +Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more here](/connect/mcp/developers/#account-connections). -### Getting started +## Getting started @@ -30,34 +30,13 @@ Now set the following environment variables (learn more about environments in Pi OPENAI_API_KEY=your_openai_api_key PIPEDREAM_CLIENT_ID=your_client_id PIPEDREAM_CLIENT_SECRET=your_client_secret -PIPEDREAM_PROJECT_ID=your_project_id -PIPEDREAM_ENVIRONMENT=development +PIPEDREAM_PROJECT_ID=your_project_id # proj_xxxxxxx +PIPEDREAM_ENVIRONMENT=development # development | production ``` ### Discover available MCP servers -Pipedream provides [{process.env.PUBLIC_APPS}+ APIs as MCP servers](https://mcp.pipedream.com) that can be used with OpenAI's tool calls. Each server corresponds to an app integration (like Notion, Gmail, or Slack) and has its own specific set of tools that you can expose to OpenAI. - - - -```javascript -// Get all available apps (paginated) -const apps = await pd.getApps(); - -// Each app has these key properties: -// - name_slug: Used in the MCP server URL (e.g., "notion", "gmail", "slack") -// - name: Display name (e.g., "Notion", "Gmail", "Slack") -``` - - -```javascript -// Search by app name -const notionApps = await pd.getApps({ q: "notion" }); -const gmailApps = await pd.getApps({ q: "gmail" }); -const slackApps = await pd.getApps({ q: "slack" }); -``` - - +[See here](/connect/mcp/developers/#discover-available-mcp-servers) for guidance on discovering the apps Pipedream has available as MCP servers. ### Generate a model response in OpenAI with Pipedream MCP @@ -120,8 +99,6 @@ console.log(response); ```bash -# Complete example from start to finish - # Step 1: Get access token from Pipedream ACCESS_TOKEN=$(curl -s -X POST https://api.pipedream.com/v1/oauth/token \ -H "Content-Type: application/json" \ @@ -164,24 +141,9 @@ curl -X POST https://api.openai.com/v1/chat/completions \ }' ``` - + -## Account Connection - -One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage. +## Account connections -You can handle account connections in one of two ways in your app: - -### Add a button in your UI -- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI -- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com) - -### Return a link -- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account -- There is no implementation required for this option since it's already handled in Pipedream's MCP server -- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example: - -``` -https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app=notion -``` \ No newline at end of file +[See here](/connect/mcp/developers/#account-connections) for more info. \ No newline at end of file From c399e6e12858404062be7948efcae0a8985cc5dc Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Tue, 20 May 2025 22:47:17 -0700 Subject: [PATCH 2/3] Update pnpm-lock.yaml --- pnpm-lock.yaml | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index ba473b3b5fc9d..1fa7a92823dd6 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -6230,8 +6230,7 @@ importers: components/hub_planner: {} - components/hubflo: - specifiers: {} + components/hubflo: {} components/hubspot: dependencies: @@ -7186,8 +7185,7 @@ importers: specifier: ^3.0.0 version: 3.0.3 - components/lead_identity_check: - specifiers: {} + components/lead_identity_check: {} components/leadboxer: {} @@ -7642,8 +7640,7 @@ importers: specifier: ^1.5.1 version: 1.6.6 - components/magileads: - specifiers: {} + components/magileads: {} components/magnetic: dependencies: @@ -7721,8 +7718,7 @@ importers: specifier: ^4.6.0 version: 4.6.0 - components/maileroo: - specifiers: {} + components/maileroo: {} components/mailersend: dependencies: @@ -8462,8 +8458,7 @@ importers: specifier: ^3.0.3 version: 3.0.3 - components/murf: - specifiers: {} + components/murf: {} components/murlist: {} @@ -10466,8 +10461,7 @@ importers: specifier: ^3.0.0 version: 3.0.3 - components/pushinator: - specifiers: {} + components/pushinator: {} components/pushover: dependencies: @@ -14007,8 +14001,7 @@ importers: specifier: ^1.2.0 version: 1.6.6 - components/uspacy: - specifiers: {} + components/uspacy: {} components/usps: {} @@ -14057,8 +14050,7 @@ importers: specifier: ^0.0.1-security version: 0.0.1-security - components/veedea: - specifiers: {} + components/veedea: {} components/vend: dependencies: From cb8effc92a35b5d5b500fa6d1bbfb02bfac236d3 Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Tue, 20 May 2025 23:15:27 -0700 Subject: [PATCH 3/3] Updates --- docs-v2/pages/connect/mcp/developers.mdx | 48 +++++++++++++----------- docs-v2/pages/connect/mcp/openai.mdx | 6 +-- 2 files changed, 28 insertions(+), 26 deletions(-) diff --git a/docs-v2/pages/connect/mcp/developers.mdx b/docs-v2/pages/connect/mcp/developers.mdx index a5ed0e7d36ad9..b6d600cabc0a4 100644 --- a/docs-v2/pages/connect/mcp/developers.mdx +++ b/docs-v2/pages/connect/mcp/developers.mdx @@ -6,7 +6,7 @@ Add Pipedream's MCP server to your application or agent to make tool calls on be ## Overview -Pipedream's MCP server code is [publicly available in GitHub](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md), and you have two options for using Pipedream's MCP server in your app: +Pipedream's MCP server code is [publicly available on GitHub](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md), and you have two options for using Pipedream's MCP server in your app: 1. [Use Pipedream's remote MCP server](#use-pipedreams-remote-mcp-server) 2. [Self-host Pipedream's MCP server](#self-host-pipedreams-mcp-server) @@ -26,25 +26,6 @@ The MCP server accepts two route params: - The app's "name slug" (the unique identifier for the app) - [See below](#discover-available-mcp-servers) for discovering available apps -### Account connections - -One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage. - -You can handle account connections in one of two ways in your app: - -### Add a button in your UI -- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI -- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com) - -### Return a link -- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account -- There is no implementation required for this option since it's already handled in Pipedream's MCP server -- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example: - -``` -https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app={appSlug} -``` - ## Getting started ### Prerequisites @@ -66,6 +47,31 @@ PIPEDREAM_PROJECT_ID=your_project_id # proj_xxxxxxx PIPEDREAM_ENVIRONMENT=development # development | production ``` +### Authentication + +#### Developer authentication + +Your application authenticates with Pipedream using client credential OAuth. [See below](#api-authentication) for details. + +#### User account connections + +One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage. + +You can handle account connections in one of two ways in your app: + +##### Add a button in your UI +- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI +- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com) + +##### Return a link +- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account +- There is no implementation required for this option since it's already handled in Pipedream's MCP server +- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example: + +``` +https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app={appSlug} +``` + ### Discover available MCP servers Pipedream provides [{process.env.PUBLIC_APPS}+ APIs as MCP servers](https://mcp.pipedream.com). Each server corresponds to an app integration (like Notion, Gmail, or Slack) and has its own specific set of tools that you can expose to OpenAI. @@ -103,7 +109,7 @@ Pipedream MCP server supports both SSE and streamable HTTP transport types, with https://mcp.pipedream.net ``` -#### Authentication +#### API Authentication To authenticate requests to Pipedream's MCP server, you need to include an access token with every HTTP request. Here's how to get it: diff --git a/docs-v2/pages/connect/mcp/openai.mdx b/docs-v2/pages/connect/mcp/openai.mdx index 390fbf1269970..0a6bcbb151ab5 100644 --- a/docs-v2/pages/connect/mcp/openai.mdx +++ b/docs-v2/pages/connect/mcp/openai.mdx @@ -5,7 +5,7 @@ import { Callout, Tabs, Steps } from 'nextra/components' Access {process.env.PUBLIC_APPS}+ APIs and 10,000+ tools in OpenAI using Pipedream Connect. MCP makes it easy to extend the capabilities of any LLM or agent, and Pipedream offers drop-in support for [calling tools in OpenAI](https://platform.openai.com/docs/guides/tools?api-mode=responses) . -Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more here](/connect/mcp/developers/#account-connections). +Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more here](/connect/mcp/developers/#user-account-connections). ## Getting started @@ -143,7 +143,3 @@ curl -X POST https://api.openai.com/v1/chat/completions \ - -## Account connections - -[See here](/connect/mcp/developers/#account-connections) for more info. \ No newline at end of file