Skip to content

Add desktop control security policy and opt-in model #461

@kovtcharov

Description

@kovtcharov

Summary

Desktop control is inherently more dangerous than browser use — it can interact with any application, see credentials on screen, and modify system state. This issue defines the security policy required before shipping CUA.

Risk Assessment

Risk Severity Mitigation
Close critical applications High App allowlist, confirmation for destructive actions
Capture credentials on screen High Credential keyword detection, screenshot auto-redaction
Delete files via file manager High Block file manager interactions by default
Rapid uncontrolled clicking Medium Rate limiting (max 60 actions/min)
Control system settings Medium Block system preferences apps
Privacy — screenshots capture everything Medium No screenshot persistence by default

Security Policy

@dataclass
class DesktopSecurityPolicy:
    # Activation
    enabled: bool = False                    # Must be explicitly enabled
    require_env_var: bool = True             # GAIA_DESKTOP_CONTROL_ENABLED=true

    # Application control
    allowed_apps: List[str] = None           # None = all (except blocked)
    blocked_apps: List[str] = [              # Never control these
        "Terminal", "iTerm", "cmd", "powershell",  # Shell access
        "System Preferences", "Settings",          # System config
        "Keychain", "Credential Manager",          # Credentials
    ]

    # Input controls
    mouse_enabled: bool = True
    keyboard_enabled: bool = True
    max_actions_per_minute: int = 60
    blocked_key_combos: List[str] = ["ctrl+alt+del", "cmd+q"]

    # Screenshot controls
    persist_screenshots: bool = False        # Don't save to disk by default
    redact_credentials: bool = True          # Blur password fields

    # Safety
    confirm_destructive: bool = True         # Confirm before closing windows
    max_session_duration_min: int = 30       # Auto-disable after 30 min
    require_user_confirmation: bool = False  # Optional: confirm each action

User Experience

  1. User runs `gaia chat --enable-desktop-control`
  2. Warning banner displayed:

    ⚠️ Desktop control enabled. The agent can see and interact with your screen.
    Only use this with trusted tasks. Press Ctrl+C at any time to stop.

  3. Agent announces actions before performing them
  4. Session auto-expires after 30 minutes

Files

  • `src/gaia/agents/chat/tools/desktop_tools.py` (security validation)
  • `tests/unit/chat/test_desktop_security.py` (NEW)
  • `docs/guides/desktop-control.mdx` (security section)

Acceptance Criteria

  • Desktop control disabled by default — requires explicit opt-in
  • Warning banner shown on activation
  • Blocked apps cannot be controlled
  • Rate limiting enforced
  • Blocked key combinations rejected
  • Auto-expiry after configured duration
  • Unit tests for every security check
  • Security policy documented

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentcuaComputer Use Agentdomain:multimodalVoice (ASR/TTS), Vision (VLM), Image gen (SD), CUAp2low prioritysecuritySecurity-sensitive changestrack:consumer-appHermes-competitor consumer product — mobile-first, voice + messaging + memory + skills

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions