Pob (Perception & Operation Bridge) is designed to connect AI with desktop applications.
It perceives the application through the window that contains it and interacts with the application.
Features:
- Macro PSL language, combines macros with AI prompts.
- MCP server, enables AI to operate desktop applications directly.
- Pob server, provides a web interface for remote control and monitoring.
- Supports Windows, macOS, and Linux.
- From the repository
Clone it and cd into the repo.
./setup.sh # select your OS, check toolchains, build core + shell
./start.sh # run it
- From a release zip
Unzip it and install — the app goes somewhere permanent
and the pob command lands on your PATH:
./install.sh # Linux
powershell -ExecutionPolicy Bypass -File install.ps1 # Windows
On macOS drag Pob.app to Applications and use Pob → Install 'pob'
Command, in the app menu.
To operate the application, Pob uses an AI native language called PSL (Prompt Script Language).
How a macro runs:
Pob executes it line by line -> reaches an AI slot -> psl fills the slot in -> execution continues
How it looks like (src/main.macro.psl):
// Reply to every unread message, then sign out.
move(398, 915)
click()
if (:: a chat window is open ::) {
loop (:: another unread message in the list ::, 10) {
move(:: the x offset to the message box ::, 738)
click()
typeText(:: a short reply to the message on screen ::)
keyPress("return")
}
}
call("sign-out.macro.psl")
| Doc | What's in it |
|---|---|
| Architecture | How the brain and the native shells are split, and how they talk |
| UI | The window and every toolbar button |
| Macro PSL | src/main.macro.psl and the Prompt Script Language it is written in: recording and replaying, every statement, and the :: … :: slots psl fills in as it runs |
| Key names | What keyPress / key_press accepts |
| Logs | The ~/.pob/ tree: the instance directory and its sessions |
| Settings | Every key in settings.json |
| CLI | The pob command |
| MCP Server | Driving Pob from Claude Code, Claude Desktop, Gemini CLI |
| Pob Server | The server every instance runs, and its address |
| Operation API | The HTTP command grammar for driving the machine |
| Control API | The localhost API the pob CLI drives the app with |
| Web UI | The remote control page, for a phone |
| Pob Keyboard | The desktop keyboard and trackpad client |
| Development | Building, the dev scripts, and cutting a release |
| Windows VM | Running and driving the Windows shell in a VM, from a Mac |
Pob License © 2026 Heyang Liu
You can fork this code and run it on your own machine for non-commercial use. Commercial use of the software, or any part of it, is not permitted — that includes use inside a company or in the course of paid work. Selling it, renamed or not, and offering any product that competes with it, free of charge or not, are not permitted either.