diff --git a/sources/platform/integrations/ai/mcp.md b/sources/platform/integrations/ai/mcp.md index 422f58e5d6..2f1f9cd64e 100644 --- a/sources/platform/integrations/ai/mcp.md +++ b/sources/platform/integrations/ai/mcp.md @@ -1,279 +1,336 @@ --- title: Apify MCP server sidebar_label: MCP -description: Learn how to use the Apify MCP server to integrate Apify Actors into your AI agents or applications. +description: Learn how to use the Apify MCP server to integrate Apify's library of Actors into your AI agents or large language model-based applications. sidebar_position: 1 slug: /integrations/mcp +toc_max_heading_level: 4 --- -The _Apify Model Context Protocol (MCP) Server_ allows AI applications to connect to Apify's extensive library of Actors as tools to perform web scraping, data extraction, or other automation tasks in real time. +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; -![Apify MCP Server](../../images/apify_mcp_server.png) - -## Quickstart - -You can use the Apify MCP Server in two ways: +The _Apify Model Context Protocol (MCP) Server_ enables AI applications to connect to Apify's extensive library of Actors. Tools allowing your AI agents to perform web scraping, data extraction, and automation tasks in real time. -- _HTTPS Endpoint_ `mcp.apify.com`: Connect your MCP client through OAuth or by including `Authorization: Bearer ` header in your requests. - - `https://mcp.apify.com` for streamable transport -- _Standard Input/Output (stdio)_: Ideal for local integrations and command-line tools such as the Claude for Desktop client. - - Set MCP client server command to `npx @apify/actors-mcp-server` and environment variable `APIFY_TOKEN` to your Apify API token - - See `npx @apify/actors-mcp-server --help` for more options +![Apify MCP Server](../../images/apify_mcp_server.png) -:::tip Quick setup options -_MCP server configuration for other clients_: Use https://mcp.apify.com to select Actors and tools, then copy the configuration to your client. +## Prerequisites -_Claude Desktop_: Download and run the [Apify MCP Server DXT file](https://github.com/apify/actors-mcp-server/releases/latest/download/apify-mcp-server.dxt) for one-click installation. +Before connecting your AI to Apify, you'll need three things: -::: +- _An Apify account_ - Sign up for an Apify account, if you don't have one. +- _Apify API token_ - Get your API token from the **Integrations** section in [Apify Console](https://console.apify.com/account#/integrations). This token authorizes the MCP server to run Actors on your behalf. Make sure to keep it secure. +- _MCP client_ - An AI agent or client that supports Model Context Protocol (MCP) This could be Anthropic's Claude for Desktop, a VS Code extension with MCP support, or any application that implements the MCP specification. The [official MCP documentation](https://modelcontextprotocol.io/clients) maintains a list of compatible clients. -## Prerequisites +## Quick start -Before you start, make sure you have the following: +You can connect to the Apify MCP server in two ways: use our hosted service for a quick and easy setup using [Streamable HTTP with OAuth](#streamable-http-with-oauth-recommended), or run the server locally for development and testing using [local stdio](#local-stdio). -1. _An Apify account:_ Sign up for a free Apify account if you don’t have one. -1. _Apify API Token:_ Get your personal API token from the **Integrations** section in [Apify Console](https://console.apify.com/account#/integrations). This token will be used to authorize the MCP server to run Actors on your behalf. -1. _MCP client:_ An AI agent or client that supports MCP. This could be Anthropic Claude for Desktop, a VS Code extension with MCP support, Apify’s web-based Tester MCP Client, or any custom client implementation. See supported MCP clients in [official documentation](https://modelcontextprotocol.io/clients). +### Streamable HTTP with OAuth (recommended) -## Example usage (Streamable HTTP with OAuth) +Provide the server URL `https://mcp.apify.com`. You will be redirected to your browser to sign in to your Apify account and approve the connection. -We recommend connecting through OAuth for a secure and simple authentication process. + + -During setup, provide the server URL `https://mcp.apify.com`. You will then be redirected to your browser to sign in to your Apify account and approve the connection. The configuration steps may vary slightly depending on your MCP client. +When you connect for the first time, you'll be redirected to your browser to sign in to Apify and authorize the connection. This OAuth flow ensures secure +authentication without exposing your API token. ```json { - "mcpServers": { - "apify": { - "url": "https://mcp.apify.com" - } - } + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com" + } + } } ``` -You can also use your Apify token directly, instead of OAuth, by setting the `Authorization: Bearer ` header in the MCP server configuration: + + + +You can also use your Apify token directly, instead of OAuth, by setting the `Authorization: Bearer ` header in the MCP server configuration. ```json { - "mcpServers": { - "apify": { - "url": "https://mcp.apify.com", - "headers": { - "Authorization": "Bearer your-apify-token" - } - } + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com", + "headers": { + "Authorization": "Bearer " + } + } } } ``` -## Example usage (local stdio with Claude for Desktop) +Replace `` with your actual Apify API token from the [Integrations section](https://console.apify.com/account#/integrations). + + + + +:::tip Quick setup options + +_MCP server configuration for other clients_: Use the [UI configuration tool](https://mcp.apify.com/) to select Actors and tools, then copy the configuration to your client. + +::: -Let’s walk through an example of using Claude for Desktop with the Apify MCP Server: +#### Client configuration -1. _Configure Claude for Desktop:_ Enable MCP servers via the **Developer settings**. You need to add an entry for the Apify MCP server. For instance, in Claude’s config file, under `mcpServers`, add an entry like: +Here's how to add the Apify MCP server to popular text editors and AI assistants: + + + + +To add Apify MCP server to Cursor: + +1. Create or open the `.cursor/mcp.json` file. +1. Add the following to the configuration file: + + + ```json { - "mcpServers": { - "actors-mcp-server": { - "command": "npx", - "args": ["-y", "@apify/actors-mcp-server"], - "env": { - "APIFY_TOKEN": "YOUR_APIFY_TOKEN" - } - } - } + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com" + } + } } ``` - This tells Claude to spawn the Apify MCP Server (via the [Actors MCP Server](https://www.npmjs.com/package/@apify/actors-mcp-server) npm package with your API token (on the first run, it will download the package automatically). + When you connect for the first time, you'll be redirected to your browser to sign in to Apify and authorize the connection. This OAuth flow ensures secure authentication without exposing your API token. -1. _Launch Claude and connect:_ After updating the config, restart Claude for Desktop. If successful, Claude will show a “plugin” (often indicated by a plug icon 🔌) signifying it connected to the Apify Actors MCP server. + + -1. _Use the Actors in conversation:_ You can chat with Claude and ask it to use Apify Actors. For example: _“What Apify Actors can I use?”_ Claude will list available tools via the MCP server. If none are pre-loaded, it may show defaults or guide you to find more. + You can also use your Apify token directly, instead of OAuth, by setting the `Authorization: Bearer ` header in the MCP server configuration. -:::note Web-based alternative + ```json + { + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com", + "headers": { + "Authorization": "Bearer " + } + } + } + } + ``` -If you prefer not to set up Claude desktop, you can achieve a similar result using [Apify’s Tester MCP Client](https://apify.com/jiri.spilka/tester-mcp-client), which provides a web UI to test the MCP server.) + Replace `` with your actual Apify API token from the [Integrations section](https://console.apify.com/account#/integrations). -::: + + -![Apify Tester MCP Client](../../images/chat-ui.webp) + + -## Interact with the MCP server over legacy SSE transport +VS Code supports MCP through GitHub Copilot's agent mode(requires Copilot subscription): -:::caution Legacy SSE transport is discouraged +1. Ensure you have GitHub Copilot installed +1. Open Command Palette (CMD/CTRL + Shift + P) and run _MCP: Open User Configuration_ command. + - This will open `mcp.json` file in your user profile. If the file does not exist, VS Code creates it for you. +1. Add the following to the configuration file: -We strongly recommend using the latest [streamable HTTP transport](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#streamable-http) for new integrations at `https://mcp.apify.com`. The legacy SSE transport at `https://mcp.apify.com/sse` is deprecated and will be removed in the future. + + + + ```json + { + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com" + } + } + } + ``` + + When you connect for the first time, you'll be redirected to your browser to sign in to Apify and authorize the connection. This OAuth flow ensures secure authentication without exposing your API token. + + + + + You can also use your Apify token directly, instead of OAuth, by setting the `Authorization: Bearer ` header in the MCP server configuration. + + ```json + { + "mcpServers": { + "apify": { + "url": "https://mcp.apify.com", + "headers": { + "Authorization": "Bearer " + } + } + } + } + ``` + + Replace `` with your actual Apify API token from the [Integrations section](https://console.apify.com/account#/integrations). + + + + + + + +:::tip One-click installation + +Download and run the [Apify MCP Server DXT file](https://github.com/apify/actors-mcp-server/releases/latest/download/apify-mcp-server.dxt) for one-click installation. ::: -You can interact with the server through legacy Server-Sent Events (SSE) to send messages and receive responses. +To manually configure Apify's MCP server for Claude Desktop: -In the client settings, you need to provide server configuration: +1. Open Claude Desktop settings. +1. Navigate to the **Developer** section. +1. Add the following to the configuration file: ```json { - "mcpServers": { - "apify": { - "type": "sse", - "url": "https://mcp.apify.com/sse", - "headers": { - "Authorization": "Bearer your-apify-token" - } - } + "mcpServers": { + "actors-mcp-server": { + "command": "npx", + "args": ["-y", "@apify/actors-mcp-server"], + "env": { + "APIFY_TOKEN": "" + } } + } } ``` -## Tools, resources, and prompts +Replace `` with your actual Apify API token from the [Integrations section](https://console.apify.com/account#/integrations). -The MCP server provides a set of tools for interacting with Apify Actors. -Since the Apify Store is large and growing rapidly, the MCP server provides a way to dynamically discover and use new Actors. + + -### Actors +### Local stdio -Any [Apify Actor](https://apify.com/store) can be used as a tool. -By default, the server is pre-configured with one Actor, `apify/rag-web-browser`, and several helper tools. -The MCP server loads an Actor's input schema and creates a corresponding MCP tool. -This allows the AI agent to know exactly what arguments to pass to the Actor and what to expect in return. +If your client doesn't support remote MCP servers using the `https://mcp.apify.com` URL, you can run the server locally instead. This method uses the stdio transport to connect directly through your local environment. -For example, for the `apify/rag-web-browser` Actor, the input parameters are: +Add this to your configuration file: ```json { - "query": "restaurants in San Francisco", - "maxResults": 3 + "mcpServers": { + "actors-mcp-server": { + "command": "npx", + "args": ["-y", "@apify/actors-mcp-server"], + "env": { + "APIFY_TOKEN": "YOUR_APIFY_TOKEN" + } + } + } } ``` -You don't need to manually specify which Actor to call or its input parameters; the LLM handles this automatically. -When a tool is called, the arguments are automatically passed to the Actor by the LLM. -You can refer to the specific Actor's documentation for a list of available arguments. +The server will download automatically on first use and connect using your API token. -### Helper tools +## Tool selection -One of the most powerful features of using MCP with Apify is dynamic tool discovery. -It gives an AI agent the ability to find new tools (Actors) as needed and incorporate them. -Here are some special MCP operations and how the Apify MCP Server supports them: +By default, the MCP server loads essential tools for Actor discovery, documentation search, and the RAG Web Browser Actor. You can customize which tools +are available by adding parameters to the server URL: -- **Apify Actors**: Search for Actors, view their details, and use them as tools for the AI. -- **Apify documentation**: Search the Apify documentation and fetch specific documents to provide context to the AI. -- **Actor runs**: Get lists of your Actor runs, inspect their details, and retrieve logs. -- **Apify storage**: Access data from your datasets and key-value stores. +`https://mcp.apify.com?tools=actors,docs,apify/rag-web-browser` -### Overview of available tools +For minimal setups where you only need specific Actors: -Here is an overview list of all the tools provided by the Apify MCP Server. +`https://mcp.apify.com?tools=apify/instagram-scraper,apify/google-search-scraper` -| Tool name | Category | Enabled by default | Description | -| :--- | :--- | :--- | :---: | -| `search-actors` | actors | ✅ | Search for Actors in the Apify Store. | -| `fetch-actor-details` | actors | ✅ | Retrieve detailed information about a specific Actor. | -| `call-actor` | actors | ✅ | Call an Actor and get its run results. | -| [`apify-slash-rag-web-browser`](https://apify.com/apify/rag-web-browser) | Actor (see [tool configuration](#tools-configuration)) | ✅ | An Actor tool to browse the web. | -| `search-apify-docs` | docs | ✅ | Search the Apify documentation for relevant pages. | -| `fetch-apify-docs` | docs | ✅ | Fetch the full content of an Apify documentation page by its URL. | -| `get-actor-run` | runs | | Get detailed information about a specific Actor run. | -| `get-actor-run-list` | runs | | Get a list of an Actor's runs, filterable by status. | -| `get-actor-log` | runs | | Retrieve the logs for a specific Actor run. | -| `get-dataset` | storage | | Get metadata about a specific dataset. | -| `get-dataset-items` | storage | | Retrieve items from a dataset with support for filtering and pagination. | -| `get-dataset-schema` | storage | | Generate a JSON schema from dataset items. | -| `get-key-value-store` | storage | | Get metadata about a specific key-value store. | -| `get-key-value-store-keys`| storage | | List the keys within a specific key-value store. | -| `get-key-value-store-record`| storage | | Get the value associated with a specific key in a key-value store. | -| `get-dataset-list` | storage | | List all available datasets for the user. | -| `get-key-value-store-list`| storage | | List all available key-value stores for the user. | -| `add-actor` | experimental | | Add an Actor as a new tool for the user to call. | - -### Tools configuration - -The `tools` configuration parameter is used to specify loaded tools - either categories or specific tools directly, and Apify Actors. For example, `tools=storage,runs` loads two categories; `tools=add-actor` loads just one tool. - -When no query parameters are provided, the MCP server loads the following `tools` by default: - -- `actors` -- `docs` -- `apify/rag-web-browser` - -If the tools parameter is specified, only the listed tools or categories will be enabled - no default tools will be included. +This configuration approach works for both hosted and local setups. For the CLI version: + +`npx @apify/actors-mcp-server --tools actors,docs,apify/web-scraper` :::tip Easy configuration -Use the [UI configurator](https://mcp.apify.com/) to configure your server, then copy the configuration to your client. +Use the UI configurator `https://mcp.apify.com/` to select your tools visually, then copy the configuration to your client. ::: -#### Configuring the hosted server +### Dynamic tool discovery -The hosted server can be configured using query parameters in the URL. For example, to load the default tools, use: +One of the most powerful features is the ability to discover and use new Actors on demand. It can search Apify Store for relevant Actors using the `search-actors` tool, inspect Actor details to understand required inputs, add the Actor as a new tool, and execute it with appropriate parameters. -```text -https://mcp.apify.com?tools=actors,docs,apify/rag-web-browser -``` +This dynamic discovery means your AI can adapt to new tasks without manual configuration. Each discovered Actor becomes immediately available for future use in the conversation. -For minimal configuration, if you want to use only a single Actor tool - without any discovery or generic calling tools, the server can be configured as follows: +### Available tools -```text -https://mcp.apify.com?tools=apify/my-actor -``` - -This setup exposes only the specified Actor (`apify/my-actor`) as a tool. No other tools will be available. - -#### Configuring the CLI +| Tool name | Category | Enabled by default | Description | +| :--- | :--- | :--- | :--- | +| `search-actors` | actors | ✅ | Search for Actors in Apify Store | +| `fetch-actor-details` | actors | ✅ | Retrieve detailed information about a specific Actor | +| `call-actor` | actors | ✅ | Call an Actor and get its run results | +| [`apify/rag-web-browser`](https://apify.com/apify/rag-web-browser) | Actor | ✅ | Browse and extract web data | +| `search-apify-docs` | docs | ✅ | Search the Apify documentation for relevant pages | +| `fetch-apify-docs` | docs | ✅ | Fetch the full content of an Apify documentation page by its URL | +| `get-actor-run` | runs | | Get detailed information about a specific Actor run | +| `get-actor-run-list` | runs | | Get a list of an Actor's runs, filterable by status | +| `get-actor-log` | runs | | Retrieve the logs for a specific Actor run | +| `get-dataset` | storage | | Get metadata about a specific dataset | +| `get-dataset-items` | storage | | Retrieve items from a dataset with support for filtering and pagination | +| `get-dataset-schema` | storage | | Generate a JSON schema from dataset items | +| `get-key-value-store` | storage | | Get metadata about a specific key-value store | +| `get-key-value-store-keys`| storage | | List the keys within a specific key-value store | +| `get-key-value-store-record`| storage | | Get the value associated with a specific key in a key-value store | +| `get-dataset-list` | storage | | List all available datasets for the user | +| `get-key-value-store-list`| storage | | List all available key-value stores for the user | +| `add-actor` | experimental | | Add an Actor as a new tool for the user to call | +| `get-actor-output`* | - | Retrieve the output from an Actor call which is not included in the output preview of the Actor tool. | ✅ | + +:::note Retrieving full output + +The `get-actor-output` tool is automatically included with any Actor-related tool, such as `call-actor`, `add-actor`, or specific Actor tools like `apify-slash-rag-web-browser`. When you call an Actor, you receive an output preview. Depending on the output format and length, the preview may contain the complete output or only a limited version to avoid overwhelming the LLM. To retrieve the full output, use the `get-actor-output` tool with the `datasetId` from the Actor call. This tool supports limit, offset, and field filtering. -The CLI can be configured using command-line flags. For example, to load the same tools as in the hosted server configuration, use: +::: -```bash -npx @apify/actors-mcp-server --tools actors,docs,apify/rag-web-browser -``` +## Advanced usage -The minimal configuration is similar to the hosted server configuration: +### Production best practices -```bash -npx @apify/actors-mcp-server --tools apify/my-actor -``` +- For production deployments, explicitly specify which tools to load rather than relying on defaults. This ensures consistent behavior across updates: -As above, this exposes only the specified Actor (`apify/my-actor`) as a tool. No other tools will be available. + `https://mcp.apify.com?tools=actors,docs,apify/rag-web-browser` -:::caution Important recommendation +- For a local stdio server, always use the latest version of the server by appending `@latest` to your npm commands. +- Monitor your API usage through Apify Console to stay within your plan limits. -_The default tools configuration may change in future versions._ When no `tools` parameter is specified, the server currently loads default tools, but this behavior is subject to change. +## Rate limits and performance -_For production use and stable interfaces, always explicitly specify the `tools` parameter_ to ensure your configuration remains consistent across updates. +The Apify MCP server allows up to _30_ requests per second per user. This limit applies to all operations including Actor runs, storage access, and +documentation queries. If you exceed this limit, you'll receive a `429` response and should implement appropriate retry logic. -::: + +## Troubleshooting -### Prompts +##### Authentication errors -The server provides a set of predefined example prompts to help you get started interacting with Apify through MCP. For example, there is a `GetLatestNewsOnTopic` prompt that allows you to easily retrieve the latest news on a specific topic using the [RAG Web Browser](https://apify.com/apify/rag-web-browser) Actor. +- _Check your API token_: Verify that your Apify API token is correct. You can find it in the **Integrations** section of the [Apify Console](https://console.apify.com/account#/integrations). Without a valid token, the server cannot start Actor runs. +- _Set environment variable for local development_: When running the MCP server locally, ensure you have set the `APIFY_TOKEN` environment variable. -### Resources +##### Local environment setup -The server does not yet provide any resources. +- _The MCP server requires Node.js v18 or higher_. Check your installed version by running `node -v` in your terminal. +- _Using the latest server version_: To ensure you have the latest features and bug fixes, use the latest version of the `@apify/actors-mcp-server` package. You can do this by appending `@latest` to the package name in your `npx` command or configuration file. -## Rate limits +##### Actor execution issues -The Apify MCP server has a rate limit of _30 requests per second_ per user. If you exceed this limit, you will receive a `429 Too Many Requests` response. +- _No response or long delays_: Actor runs can take time to complete depending on their task. If you're experiencing long delays, check the Actor's logs in Apify Console. The logs will provide insight into the Actor's status and show if it's processing a long operation or has encountered an error. -## Support and contribution + +## Support and resources -The Apify MCP Server is an open-source project. -You can report bugs, suggest new features, or ask questions in the [GitHub issues](https://github.com/apify/actors-mcp-server/issues). -If you find this project useful, please [star it on GitHub](https://github.com/apify/actors-mcp-server) to show your support! +The Apify MCP Server is an open-source project. Report bugs, suggest features, or ask questions in the [GitHub repository](https://github.com/apify/apify-mcp-server/issues). -- _Authorization (API Token):_ If the MCP server isn’t executing Actors, ensure you provided a correct Apify API token. Without a valid `APIFY_TOKEN`, the server cannot start Actor runs. Always set the `APIFY_TOKEN` environment variable when running locally. -- _Ensure latest version:_ If running via npm, always use the latest version of `@apify/actors-mcp-server` for the newest features and fixes. You can append `@latest` when installing or in your config args to ensure this. -- _Node.js environment:_ If running the server locally, make sure Node.js is installed and up to date (`node -v`). The MCP server requires Node.js v18+. -- _No response or long delay:_ Keep in mind that when an Actor tool is called, it may take some time to complete (depending on the task). If nothing is coming back, check the Actor’s logs in Apify console — the Actor might be waiting on a long operation or input. +If you find this project useful, please star it on [GitHub](https://github.com/apify/apify-mcp-server) to show your support! -## Learn more +To learn more about MCP and Apify integration: -- [Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction): Learn about the open standard on the official MCP website – understanding the protocol can help you build custom agents. -- [Actors MCP Server GitHub](https://github.com/apify/actors-mcp-server): The README for the Apify MCP Server Actor (available on Apify Store as `apify/actors-mcp-server`) provides technical details on implementation and advanced usage. -- [Apify Tester MCP Client](https://apify.com/jiri.spilka/tester-mcp-client): A specialized client Actor (`jiri.spilka/tester-mcp-client`) that you can run to simulate an AI agent in your browser. Useful for testing your setup with a chat UI. -- [How to use MCP with Apify Actors](https://blog.apify.com/how-to-use-mcp/): Learn how to expose over 5,000 Apify Actors to AI agents with Claude and LangGraph, and configure MCP clients and servers. -- [Apify MCP Server Tutorial](https://www.youtube.com/watch?v=BKu8H91uCTg): Integrate thousands of Apify Actors and Agents with Claude. +- [Model Context Protocol specification](https://modelcontextprotocol.io) - Learn about the open standard on the official MCP website – understanding the protocol can help you build custom agents. +- [How to use MCP with Apify Actors](https://blog.apify.com/how-to-use-mcp/) - Learn how to expose over thousands of Apify Actors to AI agents with Claude and LangGraph, and configure MCP clients and servers. +- [Video tutorial](https://www.youtube.com/watch?v=BKu8H91uCTg) - Integrate thousands of Apify Actors and Agents with Claude. +- [Apify Tester MCP Client](https://apify.com/jiri.spilka/tester-mcp-client) - A specialized client Actor that you can run to simulate an AI agent in your browser. Useful for testing your setup with a chat UI.