Skip to content

Computer Autopilot v1.0.3

Latest
Latest

Choose a tag to compare

View all tags
@ngotrantronghieu ngotrantronghieu released this 12 Oct 04:31
· 2 commits to main since this release
v1.0.3
684608d

Computer Autopilot v1.0.3 - MCP Integration & Window Selector Release

🎉 Major New Features

1. Window Selector for Targeted Automation

Computer Autopilot now includes a Window Selector feature that allows you to specify exactly which window the assistant should work on. No more manually switching windows - just select your target window from a dropdown and let the assistant handle the rest!

🪟 Window Selector Features

  • Dropdown Window Selector in Main Chat interface

    • Located between the input field and control buttons
    • Shows all currently open windows with application icons
    • Automatically refreshes when opened

  • Auto-detect Mode (Default)

    • Works on the currently active window
    • Perfect for quick tasks
    • No configuration needed

  • Specific Window Selection

    • Choose exactly which window to target
    • Visual feedback with green highlight when selected
    • Window is automatically focused before task execution

  • Minimized Window Support

    • All minimized windows appear in the list
    • Marked with [Minimized] indicator for easy identification
    • Automatically restored and focused when selected

  • Smart Window Filtering

    • Filters out system windows and background processes
    • Shows only relevant application windows
    • Excludes the assistant itself for cleaner list

How to Use Window Selector

  1. Click the window selector dropdown (shows "🎯 Auto-detect" by default)
  2. Select a window from the list:
    • Auto-detect - Works on currently active window
    • [Window Name] - Targets specific window
    • [Window Name] [Minimized] - Minimized window (will be restored)
  3. Type your command in the input field
  4. Press Enter - The assistant will focus the selected window and execute your task

Example Use Cases

  • Type in a specific Notepad window while working in another app
  • Click buttons in a minimized browser without manually restoring it
  • Automate tasks in background applications without losing focus on your current work
  • Switch between multiple instances of the same application easily

Visual Indicators

  • White background: Auto-detect mode (default)
  • Green background: Specific window selected
  • [Minimized] tag: Window is currently minimized
  • Window icons: Application icons displayed when available

2. Model Context Protocol (MCP) Support

Computer Autopilot now supports the Model Context Protocol (MCP), enabling the AI assistant to connect to external tools and services through a standardized protocol. This opens up endless possibilities for extending the assistant's capabilities beyond built-in Windows automation.

What is MCP?

The Model Context Protocol is a standardized way for AI applications to interact with external tools, APIs, and data sources. Think of it as a universal plugin system for AI assistants - instead of hardcoding every possible integration, MCP allows you to connect to any compatible server and instantly gain access to its tools.

Key Features

🔧 MCP Servers Management

  • Dedicated MCP Servers Tab in Settings for easy configuration
  • Visual server status monitoring with automatic health checks
  • Tool discovery and metadata display - see all available tools at a glance
  • Background status checks - non-blocking UI for testing multiple servers
  • Auto-refresh on settings open - servers are tested automatically when you open the MCP tab

🤖 Seamless AI Integration

  • Both Main Assistant and Web Agent can call MCP tools
  • Natural language tool invocation - just ask the assistant to use a tool
  • Full MCP result history preserved in assistant context for better reasoning
  • Detailed tool descriptions injected into prompts for accurate tool selection
  • Pure-JSON action format for reliable tool calling

⚙️ Easy Configuration

  • JSON-based server configuration with visual editor
  • Environment variable support for API keys and credentials
  • Stdio transport - works with any MCP server using standard input/output
  • Automatic tool metadata caching for faster prompt builds
  • Live tool discovery with fallback to configured allowlists

Example Use Cases

With MCP integration, Computer Autopilot can now:

  • Search the web using search MCP server
  • Fetch and parse web content using fetch MCP servers
  • Access filesystems with proper permissions via filesystem MCP server
  • Query databases through database MCP servers
  • Call external APIs via custom MCP server implementations
  • And much more - any MCP-compatible server works out of the box!

How to Use

  1. Configure MCP Servers in Settings → MCP Servers tab:

    • Click "Edit Servers JSON..."
    • Add your server configurations (see examples in README.md)
    • Click "Refresh Status" to test connections

  2. Use MCP Tools via natural language:

    • Main Assistant: "Search the web for latest AI news using MCP"
    • Web Agent: "Use MCP to fetch content from example.com"

  3. Monitor Results:

    • MCP tool outputs appear in the chat
    • Full result history is preserved in assistant context
    • Recent results are shown in the MCP Servers tab

Technical Details

Architecture

  • Async MCP SDK integration with synchronous wrappers for compatibility
  • Isolated event loops to avoid conflicts with Qt's event loop
  • Global result caching for efficient context building
  • Thread-safe UI updates using QTimer for background operations

Prompt Context

  • MCP tool details: Full tool names and descriptions per server
  • Recent MCP results: Complete history of all MCP calls (untruncated)
  • Automatic metadata discovery: Tools are discovered and cached on first use

Known Limitations

  • Stdio transport only: Currently supports MCP servers using stdio (standard input/output). HTTP/SSE transport coming in future releases.
  • Windows-specific paths: MCP server commands must use Windows-compatible paths and executables.
  • Node.js dependency: Many popular MCP servers require Node.js/npx to be installed.

Migration Guide

If you're upgrading from v1.0.2:

  1. No action required - MCP support is optional
  2. To enable MCP:
    • Configure servers in Settings → MCP Servers tab
    • Start using MCP tools via natural language

Popular MCP Servers

Get started quickly with these popular MCP servers:

  • Brave Search: Web search capabilities

    • npx -y @modelcontextprotocol/server-brave-search

  • Fetch: Web content retrieval and parsing

    • npx -y @lmcc-dev/mult-fetch-mcp-server

  • Filesystem: Secure file system access

    • npx -y @modelcontextprotocol/server-filesystem <allowed-path>

  • GitHub: Repository and issue management

    • npx -y @modelcontextprotocol/server-github

For a complete list, visit: https://github.com/modelcontextprotocol/servers

Credits

Feedback & Support

Full Changelog: v1.0.2...v1.0.3

Assets 3
Loading