RFC: an MCP server for Crawlee Cloud — let AI agents drive your scrapers

[aminembarki]

I want Crawlee Cloud instances to be first-class tools for AI agents. The Model Context Protocol is how Claude, Cursor, and a growing list of agent runtimes talk to external systems — and "run this scraper, wait for it, hand me the dataset" is about as natural an agent workflow as it gets.

The proposal: a platform MCP server. One server, built in TypeScript inside the existing monorepo (as a new package or a crc mcp subcommand — genuinely undecided, tell me which you'd prefer), targeting the platform exactly as it is today. It would expose tools along the lines of:

• run-actor — start a run with input, optionally wait for completion
• get-run-status — check a run's state
• get-dataset-items — fetch results as JSON
• stream-logs — tail a run's logs

Everything talks to the same Apify-compatible v2 API the CLI uses, authenticated with your existing API key. Nothing about the platform changes; this is a new front door, not a new engine. It does not depend on the Python support RFC — it works with today's TypeScript actors from day one.

For a possible layer two: Apify's own actors-mcp-server exposes each actor as its own MCP tool, with parameters derived from the actor's input schema — an agent calls scrape-amazon directly instead of a generic run-actor. That's attractive but heavier; I'd rather ship the platform server first and layer this on if people want it.

What I need from you:

1. Which tools would you actually use? Is the list above right? Missing something (trigger a schedule, push to a request queue, check platform health)?
2. Transport: stdio or remote? stdio (agent spawns the server locally, e.g. via npx) is simplest; a remote/HTTP endpoint on your instance means any agent can connect without local setup but raises the auth stakes. Which fits your deployment?
3. Auth expectations for self-hosted. Reuse your existing API key? Scoped read-only keys first (they're already on the roadmap)? What would you be comfortable pointing an agent at?

Ground rules:

• The roadmap's non-goals still apply — this is and stays a self-hosted platform.
• "I need X because Y happened to me" beats "it would be cool if X". Real friction wins.

This is on the roadmap as Proposed — RFC open: no version commitment until this discussion converges. Would you point an agent at your instance — and what would it need to do?