The agent seat: sandboxed apps an agent drives with its own pointer and keyboard

[KewkLW]

Context: #10688 asks for a human and an agent on one Hyprland session with independent cursors and focus. #11342 proposes Cua for Install > AI, which drives apps through a Hyprland plugin. This is a different piece: a seat for the agent, built in userspace, with the apps it drives sandboxed.

What it is. omarchy agent seat run chromium https://omarchy.org starts the app in a bubblewrap sandbox connected to a small Wayland proxy instead of Hyprland. The window is a real window in your session, on its own workspace (10 by default) with an orange outline. The agent clicks and types in it with a pointer and keyboard of its own, while yours stay where they are; you can use the same windows with your own mouse and keyboard at any time, for a login or a captcha (Super + Ctrl + Alt + A jumps there, and Trigger > Agents > Agent Seat is always in the menu, opening your default browser on the seat when nothing is on it). No Hyprland plugin, no Hyprland changes.

How agents see. Names first, pixels second. omarchy agent seat ui <window> lists an app's buttons, fields and links from its accessibility tree (GTK and Qt over a private bus per sandbox; Chromium and Electron through DevTools with --cdp), click-ui <window> Save clicks by name, and state says whether you or the agent acted last. Screenshots are the fallback, composed from the apps' own buffers, popups included, even while the workspace is hidden.

What the sandbox covers. Agent input reaches only agent apps. The apps see an allowlist of Wayland globals: no input injection, screen capture, window lists, overlays, clipboard snooping or GPU. They get their own home and bus, none of your files unless the agent shares a folder, and no keyring, portals, Hyprland socket or X11. Your clipboard works in them for you, but the agent can't read it or set it.

What it doesn't cover. The network is shared, so a page in an agent app can reach services on your localhost. An app can read your clipboard while your keyboard is in it and you acted last, so a hostile page sees it once you click in. It's bubblewrap, not a VM. And it boxes the apps, not the agent, which still runs as you.

Not the same as vimalinx/AgentSeat (same name, a nested compositor per app, no sandbox) or Cua's Hyprland plugin (pinned to the compositor build, unsandboxed apps). It's meant as the sandboxed complement to tools like Cua, not a replacement.

Status. A pull request is ready with the broker, the sandbox, the window rules, ui/click-ui/state, the menu entries, the skill topic, the manual section and tests (69 checks across five suites), and a reference document, docs/agent-seat.md. GPU access behind a consent prompt, and driving your own windows under a policy, are built and come later as separate PRs.

A recording will follow.

Read more: docs/agent-seat.md on the branch covers how it works, the decisions and what was rejected, the problems met, its limits, how it compares with other approaches, and a FAQ.

Written with AI coding agents (Claude Code), directed, reviewed and tested by me on my own install.

Pull request: #15087

🤖 Generated with Claude Code