🤖 Automation MCP

Automation MCP is a Model Context Protocol (MCP) server that provides AI models with complete desktop automation capabilities on macOS. It enables AI assistants to:

🖱️ Control your mouse (click, move, scroll, drag)
⌨️ Type and send keyboard input (including system shortcuts)
📸 Take screenshots and analyze screen content
🪟 Manage windows (focus, move, resize, minimize)
🎯 Interact with UI elements through coordinates
🎨 Analyze screen colors and highlight regions
🔍 Wait for images to appear on screen

🚀 Quick start

Make sure you have furi installed, and then run the following command:

furi add ashwwwin/automation-mcp

followed by:

furi start ashwwwin/automation-mcp

and you're done! (or you can just use the furi desktop app for no cli).

🥲 Normal start (without furi)

Prerequisites

Bun runtime - Install with: curl -fsSL https://bun.sh/install | bash

1. Clone and Install

git clone https://github.com/ashwwwin/automation-mcp.git
cd automation-mcp
bun install

2. Start the Server

# Start with HTTP transport (recommended for web apps)
bun run index.ts

# Or start with stdio transport (for command line tools)
bun run index.ts --stdio

3. Grant Permissions

On first run, macOS will ask for permissions. You must grant these for full functionality:

Accessibility - Allows keyboard/mouse control
Screen Recording - Enables screenshots and screen analysis

Or manually enable in: System Settings → Privacy & Security → Accessibility/Screen Recording

🛠️ Available Tools

🖱️ Mouse Control

mouseClick - Click at coordinates with left/right/middle button
mouseDoubleClick - Double-click at coordinates
mouseMove - Move cursor to position
mouseGetPosition - Get current cursor location
mouseScroll - Scroll in any direction
mouseDrag - Drag from current position to target
mouseButtonControl - Press/release mouse buttons
mouseMovePath - Follow a smooth path with multiple points

⌨️ Keyboard Input

type - Type text or press key combinations
keyControl - Advanced key press/release control
systemCommand - Common shortcuts (copy, paste, undo, save, etc.)

📸 Screen Capture & Analysis

screenshot - Capture full screen, regions, or specific windows
screenInfo - Get screen dimensions
screenHighlight - Highlight screen regions visually
colorAt - Get color of any pixel
waitForImage - Wait for images to appear (template matching)

🪟 Window Management

getWindows - List all open windows
getActiveWindow - Get current active window
windowControl - Focus, move, resize, minimize windows

🔒 Security & Permissions

Accessibility - Required for:
- Mouse clicks and movement
- Keyboard input simulation
- Window management
Screen Recording - Required for:
- Taking screenshots
- Screen analysis
- Color detection

🚀 Integration Examples

With Claude Desktop + furi

If you've already configured furi with Claude Desktop, you don't need to do anything.

Add to your MCP configuration:

{
  "mcpServers": {
    "furi": {
      "command": "furi",
      "args": ["connect"]
    }
  }
}

With Claude Desktop

Add to your MCP configuration:

{
  "mcpServers": {
    "automation": {
      "command": "bun",
      "args": ["run", "/path/to/automation-mcp/index.ts", "--stdio"]
    }
  }
}

🐛 Troubleshooting

Common Issues

Permission Denied Errors

Ensure Accessibility and Screen Recording permissions are granted
Ensure Xcode Command Line Tools: xcode-select --install

🙋‍♂️ Support

Having issues? Check the troubleshooting section above or open an issue with:

Your operating system and version
Error messages
Steps to reproduce

License

ashwwwin/automation-mcp

ErrorLooks like something went wrong!

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages