Skip to content

Danny/mcp hosted servers #16743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 21, 2025
Merged

Danny/mcp hosted servers #16743

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
77030e8
Adding info for remote servers
dannyroosevelt May 21, 2025
c399e6e
Update pnpm-lock.yaml
dannyroosevelt May 21, 2025
cb8effc
Updates
dannyroosevelt May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 185 additions & 21 deletions docs-v2/pages/connect/mcp/developers.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,42 +1,190 @@
import { Steps } from 'nextra/components'
import { Callout, Steps, Tabs } from 'nextra/components'

# Deploy Pipedream MCP to your agent
Deploy Pipedream's MCP servers to your application or agent and make authenticated request on behalf of your customers. Refer to our reference implementation [here](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md).
# Add Pipedream MCP to your app or agent

## Quickstart
Add Pipedream's MCP server to your application or agent to make tool calls on behalf of your users to {process.env.PUBLIC_APPS}+ APIs and 10,000+ tools.

<Steps>
## Overview

#### Prerequisites
Pipedream's MCP server code is [publicly available on GitHub](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md), and you have two options for using Pipedream's MCP server in your app:

To run your own MCP server, you'll need:
1. [Use Pipedream's remote MCP server](#use-pipedreams-remote-mcp-server)
2. [Self-host Pipedream's MCP server](#self-host-pipedreams-mcp-server)

### Pipedream concepts to understand

The MCP server accepts two route params:

**`external_user_id`**

- This is your user’s ID, in your system: whatever you use to uniquely identify them
- Any requests made to that route are coupled to that end user and uses the connected account tied to that user
- [See here](/connect/api/#external-users) for more detail

**`app`**

- The app's "name slug" (the unique identifier for the app)
- [See below](#discover-available-mcp-servers) for discovering available apps

## Getting started

### Prerequisites

For either option, you'll need:

1. A [Pipedream account](https://pipedream.com/auth/signup)
2. A [Pipedream project](/projects/#creating-projects). Accounts connected via MCP will be stored here.
3. [Pipedream OAuth credentials](/rest-api/auth/#oauth)

#### Set up environment variables
#### Set up your environment

Set the following environment variables:

```bash
PIPEDREAM_CLIENT_ID=your_client_id
PIPEDREAM_CLIENT_SECRET=your_client_secret
PIPEDREAM_PROJECT_ID=your_project_id
PIPEDREAM_ENVIRONMENT=development
PIPEDREAM_PROJECT_ID=your_project_id # proj_xxxxxxx
PIPEDREAM_ENVIRONMENT=development # development | production
```

### Authentication

#### Developer authentication

Your application authenticates with Pipedream using client credential OAuth. [See below](#api-authentication) for details.

#### User account connections

One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage.

You can handle account connections in one of two ways in your app:

##### Add a button in your UI
- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI
- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com)

##### Return a link
- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account
- There is no implementation required for this option since it's already handled in Pipedream's MCP server
- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example:

```
https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app={appSlug}
```

### Discover available MCP servers

Pipedream provides [{process.env.PUBLIC_APPS}+ APIs as MCP servers](https://mcp.pipedream.com). Each server corresponds to an app integration (like Notion, Gmail, or Slack) and has its own specific set of tools that you can expose to OpenAI.

<Tabs items={['List all apps', 'Search for a specific app']}>
<Tabs.Tab>
```javascript
// Get all available apps (paginated)
const apps = await pd.getApps();

// Each app has these key properties:
// - name_slug: Used in the MCP server URL (e.g., "notion", "gmail", "slack")
// - name: Display name (e.g., "Notion", "Gmail", "Slack")
```
</Tabs.Tab>
<Tabs.Tab>
```javascript
// Search by app name
const notionApps = await pd.getApps({ q: "notion" });
const gmailApps = await pd.getApps({ q: "gmail" });
const slackApps = await pd.getApps({ q: "slack" });
```
</Tabs.Tab>
</Tabs>

### Use Pipedream's remote MCP server

<Callout type="info">
Pipedream MCP server supports both SSE and streamable HTTP transport types, with no configuration required by the developer or MCP client.
</Callout>

#### Base URL

```
https://mcp.pipedream.net
```

#### API Authentication

To authenticate requests to Pipedream's MCP server, you need to include an access token with every HTTP request. Here's how to get it:

#### Run the MCP server
<Tabs items={['Node.js', 'cURL']}>
<Tabs.Tab>
```javascript
import { createBackendClient } from "@pipedream/sdk/server";

Pipedream's MCP servers can run in two modes:
// Initialize the Pipedream client with the SDK
const pd = createBackendClient({
environment: PIPEDREAM_ENVIRONMENT,
credentials: {
clientId: PIPEDREAM_CLIENT_ID,
clientSecret: PIPEDREAM_CLIENT_SECRET,
},
projectId: PIPEDREAM_PROJECT_ID
});

##### Stdio (for local testing)
// Get access token for MCP server auth
const accessToken = await pd.rawAccessToken();

console.log(accessToken);
```
Comment on lines +119 to +135
Copy link
Contributor

@coderabbitai coderabbitai bot May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix Node.js snippet to read from process.env and handle await correctly
As written, the snippet references bare identifiers (PIPEDREAM_*) which will cause ReferenceError. Also, top-level await requires an ES module context. Proposed diff:

-import { createBackendClient } from "@pipedream/sdk/server";
-
-// Initialize the Pipedream client with the SDK
-const pd = createBackendClient({
-  environment: PIPEDREAM_ENVIRONMENT,
-  credentials: {
-    clientId: PIPEDREAM_CLIENT_ID,
-    clientSecret: PIPEDREAM_CLIENT_SECRET,
-  },
-  projectId: PIPEDREAM_PROJECT_ID
-});
-
-// Get access token for MCP server auth
-const accessToken = await pd.rawAccessToken();
-
-console.log(accessToken);
+import { createBackendClient } from "@pipedream/sdk/server";
+
+// Example assumes ES module with top‐level await enabled
+const pd = createBackendClient({
+  environment: process.env.PIPEDREAM_ENVIRONMENT,
+  credentials: {
+    clientId: process.env.PIPEDREAM_CLIENT_ID,
+    clientSecret: process.env.PIPEDREAM_CLIENT_SECRET,
+  },
+  projectId: process.env.PIPEDREAM_PROJECT_ID,
+});
+
+// Retrieve and log the access token
+const accessToken = await pd.rawAccessToken();
+console.log(accessToken);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import { createBackendClient } from "@pipedream/sdk/server";
Pipedream's MCP servers can run in two modes:
// Initialize the Pipedream client with the SDK
const pd = createBackendClient({
environment: PIPEDREAM_ENVIRONMENT,
credentials: {
clientId: PIPEDREAM_CLIENT_ID,
clientSecret: PIPEDREAM_CLIENT_SECRET,
},
projectId: PIPEDREAM_PROJECT_ID
});
##### Stdio (for local testing)
// Get access token for MCP server auth
const accessToken = await pd.rawAccessToken();
console.log(accessToken);
```
import { createBackendClient } from "@pipedream/sdk/server";
// Example assumes ES module with top-level await enabled
const pd = createBackendClient({
environment: process.env.PIPEDREAM_ENVIRONMENT,
credentials: {
clientId: process.env.PIPEDREAM_CLIENT_ID,
clientSecret: process.env.PIPEDREAM_CLIENT_SECRET,
},
projectId: process.env.PIPEDREAM_PROJECT_ID,
});
// Retrieve and log the access token
const accessToken = await pd.rawAccessToken();
console.log(accessToken);
🤖 Prompt for AI Agents 
In docs-v2/pages/connect/mcp/developers.mdx around lines 119 to 135, the Node.js
snippet incorrectly uses bare identifiers for environment variables and
top-level await without an async context. Fix this by replacing all
`PIPEDREAM_*` variables with `process.env.PIPEDREAM_*` to correctly access
environment variables. Also, wrap the asynchronous code inside an async function
or use an immediately invoked async function expression to properly handle the
await keyword.

</Tabs.Tab>
<Tabs.Tab>
```bash
npx @pipedream/mcp stdio --app slack --external-user-id user123
curl -s -X POST https://api.pipedream.com/v1/oauth/token \
-H "Content-Type: application/json" \
-d '{
"grant_type": "client_credentials",
"client_id": "'$PIPEDREAM_CLIENT_ID'",
"client_secret": "'$PIPEDREAM_CLIENT_SECRET'"
}'
```
Comment on lines +139 to +146
Copy link
Contributor

@coderabbitai coderabbitai bot May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Correct quoting in the cURL -d payload
Single-quoted strings in Bash do not expand variables. Change to double quotes and escape inner quotes so ${PIPEDREAM_*} is interpolated correctly:

-curl -s -X POST https://api.pipedream.com/v1/oauth/token \
-  -H "Content-Type: application/json" \
-  -d '{
-    "grant_type": "client_credentials", 
-    "client_id": "'$PIPEDREAM_CLIENT_ID'", 
-    "client_secret": "'$PIPEDREAM_CLIENT_SECRET'"
-  }'
+curl -s -X POST https://api.pipedream.com/v1/oauth/token \
+  -H "Content-Type: application/json" \
+  -d "{
+    \"grant_type\": \"client_credentials\",
+    \"client_id\": \"${PIPEDREAM_CLIENT_ID}\",
+    \"client_secret\": \"${PIPEDREAM_CLIENT_SECRET}\"
+  }"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
curl -s -X POST https://api.pipedream.com/v1/oauth/token \
-H "Content-Type: application/json" \
-d '{
"grant_type": "client_credentials",
"client_id": "'$PIPEDREAM_CLIENT_ID'",
"client_secret": "'$PIPEDREAM_CLIENT_SECRET'"
}'
```
curl -s -X POST https://api.pipedream.com/v1/oauth/token \
-H "Content-Type: application/json" \
-d "{
\"grant_type\": \"client_credentials\",
\"client_id\": \"${PIPEDREAM_CLIENT_ID}\",
\"client_secret\": \"${PIPEDREAM_CLIENT_SECRET}\"
}"
🤖 Prompt for AI Agents 
In docs-v2/pages/connect/mcp/developers.mdx around lines 139 to 146, the cURL
command uses single quotes around the JSON payload, preventing Bash from
expanding the environment variables. Change the outer quotes to double quotes
and escape the inner double quotes properly so that the ${PIPEDREAM_CLIENT_ID}
and ${PIPEDREAM_CLIENT_SECRET} variables are correctly interpolated in the
payload.

</Tabs.Tab>
</Tabs>

#### Required headers

Include these headers in every HTTP request:

```javascript
{
Authorization: `Bearer ${PIPEDREAM_ACCESS_TOKEN}`,
"x-pd-project-id": PIPEDREAM_PROJECT_ID, // proj_xxxxxxx
"x-pd-environment": PIPEDREAM_ENVIRONMENT // development | production
}
Comment on lines +155 to +159
Copy link
Contributor

@coderabbitai coderabbitai bot May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Clarify header variable references
The JavaScript snippet shows bare PIPEDREAM_ACCESS_TOKEN, PIPEDREAM_PROJECT_ID, etc., which aren’t defined. Update to use the accessToken variable and read env vars via process.env:

-{
-  Authorization: `Bearer ${PIPEDREAM_ACCESS_TOKEN}`,
-  "x-pd-project-id": PIPEDREAM_PROJECT_ID,
-  "x-pd-environment": PIPEDREAM_ENVIRONMENT // development | production
-}
+{
+  Authorization: `Bearer ${accessToken}`,
+  "x-pd-project-id": process.env.PIPEDREAM_PROJECT_ID,
+  "x-pd-environment": process.env.PIPEDREAM_ENVIRONMENT, // development | production
+}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
{
Authorization: `Bearer ${PIPEDREAM_ACCESS_TOKEN}`,
"x-pd-project-id": PIPEDREAM_PROJECT_ID, // proj_xxxxxxx
"x-pd-environment": PIPEDREAM_ENVIRONMENT // development | production
}
{
Authorization: `Bearer ${accessToken}`,
"x-pd-project-id": process.env.PIPEDREAM_PROJECT_ID,
"x-pd-environment": process.env.PIPEDREAM_ENVIRONMENT, // development | production
}
🤖 Prompt for AI Agents 
In docs-v2/pages/connect/mcp/developers.mdx around lines 155 to 159, the code
snippet uses undefined variables like PIPEDREAM_ACCESS_TOKEN and
PIPEDREAM_PROJECT_ID directly. Update the snippet to use the variable
accessToken for the token and access environment variables via process.env for
project ID and environment, ensuring the example is clear and functional.

```

##### SSE (for hosting)
### Self-host Pipedream's MCP server

#### Using the `Dockerfile`

If you have Docker installed locally, you can build and run the container from the [reference implementation](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/Dockerfile):

```console
> docker build -t pipedream-connect .
> docker run -d --name pd-mcp -p 3010:3010 --env-file .env pipedream-connect:latest
```

This exposes a generic MCP server at [http://localhost:3010/:external_user_id/:app](http://localhost:3010/:external_user_id/:app).

#### Start the server with Streamable HTTP Transport

```bash
pnpm dev:http
```

You can use the optional env var `PD_SDK_DEBUG` to print out all the requests and responses going to the Connect API:

```bash
PD_SDK_DEBUG=true pnpm dev:http
```

#### Start the server with SSE

```bash
npx @pipedream/mcp sse
Expand All @@ -46,13 +194,9 @@ This exposes routes at:
- `GET /:external_user_id/:app` - App-specific SSE connection endpoint
- `POST /:external_user_id/:app/messages` - App-specific message handler

Where:
- `:external_user_id` is your user's unique identifier in your system
- `:app` is the app slug (e.g., "slack", "github")

#### Customize for your needs

You can host and customize the MCP server for your specific requirements:
You can also host and customize the MCP server for your specific requirements:

```bash
# Clone the repo
Expand All @@ -71,4 +215,24 @@ npm run start:sse:prod

See the [MCP server README](https://github.com/PipedreamHQ/pipedream/blob/master/modelcontextprotocol/README.md) for detailed instructions on customization options.

</Steps>
### Use the MCP inspector

The [MCP inspector](https://modelcontextprotocol.io/docs/tools/inspector) can be helpful when debugging tool calls.

```bash
npx @modelcontextprotocol/inspector
```

Enter the server URL:

If using Pipedream's remote server:

```
https://mcp.pipedream.net/{external_user_id}/{app_slug}
```

If running locally:

```
http://localhost:3010/{external_user_id}/{app_slug}
```
54 changes: 6 additions & 48 deletions docs-v2/pages/connect/mcp/openai.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import { Callout, Tabs, Steps } from 'nextra/components'
Access {process.env.PUBLIC_APPS}+ APIs and 10,000+ tools in OpenAI using Pipedream Connect. MCP makes it easy to extend the capabilities of any LLM or agent, and Pipedream offers drop-in support for [calling tools in OpenAI](https://platform.openai.com/docs/guides/tools?api-mode=responses) .

<Callout type="info">
Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more below](#account-connection).
Pipedream Connect includes built-in user authentication for [every MCP server](https://mcp.pipedream.com), which means you don't need to build any authorization flows or deal with token storage and refresh in order to make authenticated requests on behalf of your users. [Learn more here](/connect/mcp/developers/#user-account-connections).
</Callout>

### Getting started
## Getting started

<Steps>

Expand All @@ -30,34 +30,13 @@ Now set the following environment variables (learn more about environments in Pi
OPENAI_API_KEY=your_openai_api_key
PIPEDREAM_CLIENT_ID=your_client_id
PIPEDREAM_CLIENT_SECRET=your_client_secret
PIPEDREAM_PROJECT_ID=your_project_id
PIPEDREAM_ENVIRONMENT=development
PIPEDREAM_PROJECT_ID=your_project_id # proj_xxxxxxx
PIPEDREAM_ENVIRONMENT=development # development | production
```

### Discover available MCP servers

Pipedream provides [{process.env.PUBLIC_APPS}+ APIs as MCP servers](https://mcp.pipedream.com) that can be used with OpenAI's tool calls. Each server corresponds to an app integration (like Notion, Gmail, or Slack) and has its own specific set of tools that you can expose to OpenAI.

<Tabs items={['List all apps', 'Search for a specific app']}>
<Tabs.Tab>
```javascript
// Get all available apps (paginated)
const apps = await pd.getApps();

// Each app has these key properties:
// - name_slug: Used in the MCP server URL (e.g., "notion", "gmail", "slack")
// - name: Display name (e.g., "Notion", "Gmail", "Slack")
```
</Tabs.Tab>
<Tabs.Tab>
```javascript
// Search by app name
const notionApps = await pd.getApps({ q: "notion" });
const gmailApps = await pd.getApps({ q: "gmail" });
const slackApps = await pd.getApps({ q: "slack" });
```
</Tabs.Tab>
</Tabs>
[See here](/connect/mcp/developers/#discover-available-mcp-servers) for guidance on discovering the apps Pipedream has available as MCP servers.

### Generate a model response in OpenAI with Pipedream MCP

Expand Down Expand Up @@ -120,8 +99,6 @@ console.log(response);
</Tabs.Tab>
<Tabs.Tab>
```bash
# Complete example from start to finish

# Step 1: Get access token from Pipedream
ACCESS_TOKEN=$(curl -s -X POST https://api.pipedream.com/v1/oauth/token \
-H "Content-Type: application/json" \
Expand Down Expand Up @@ -164,24 +141,5 @@ curl -X POST https://api.openai.com/v1/chat/completions \
}'
```
</Tabs.Tab>
</Tabs>
</Tabs>
</Steps>

## Account Connection

One of the core features of Pipedream Connect and our MCP product is the ability for your users to easily connect their accounts without having to build any of the authorization flow or handle token storage.

You can handle account connections in one of two ways in your app:

### Add a button in your UI
- Use Pipedream's [frontend SDK](/connect/managed-auth/quickstart/#use-the-pipedream-sdk-in-your-frontend) to let users connect their account directly in your UI
- You can see an example of this when you connect any account in [mcp.pipedream.com](https://mcp.pipedream.com)

### Return a link
- Use [Connect Link ](/connect/managed-auth/quickstart/#or-use-connect-link) to let your users open a Pipedream hosted page to connect their account
- There is no implementation required for this option since it's already handled in Pipedream's MCP server
- If a user doesn't have a connected account that's required for a given tool call, we'll return a URL in the tool call response. For example:

```
https://pipedream.com/_static/connect.html?token=ctok_xxxxxxx&connectLink=true&app=notion
```
Loading
Loading