An OS-level AI layer for Windows: press a global hotkey in any application, type an instruction, and a pi agent already knows what you were looking at and has tools to act on it.
Notepad, Explorer, Excel, browser, anything
|
Ctrl+Alt+Space
|
v
+---------------------------+ HTTP/JSON +---------------------------+
| C# host (hidden WPF) | -------------------> | node-harness |
| - pins context BEFORE | POST /invoke | - pi SDK AgentSession |
| the popup shows | <------------------- | - desktop tools |
| - prompt overlay | POST /tools/{name} | - observe/act loops |
| - screenshots, UIA, input| | via pi extensions |
+---------------------------+ +---------------------------+
The core split:
- C# host = stable native substrate: hotkey, context pinning, screenshots, UI Automation, input primitives.
- node-harness = the intelligence: a real pi session that reasons, picks tools, and re-observes after actions.
- Everything rapidly changing lives on the node side or in sidecars — the host stays small.
| Path | Purpose |
|---|---|
host-dotnet/ |
C# solution: WindowsHarness.Host (WPF background app) + WindowsHarness.Contracts (shared schema types) |
node-harness/ |
Node service: HTTP server on port 17832, pi SDK agent sessions, desktop tool wrappers |
shared/schemas/ |
Canonical TypeScript types for the context snapshot (desktop-context.ts) |
shared/protocol/ |
The localhost HTTP contract (protocol.md) — ports, endpoints, error model |
See DEVELOPMENT.md for prerequisites, build and test
commands, development workflows, environment variables, and instructions for
refreshing the installed application.
- Focus any desktop app (Notepad, Explorer, ...).
- Press Ctrl+Alt+Space.
- Type an instruction, press Enter. The overlay closes instantly and focus returns to your app.
- The harness logs show the pinned snapshot, then the agent's answer.
A pi session receives the context summary and screenshot. It can observe the
captured window and use desktop_act to focus, click, type, press supported
keys or shortcuts, and scroll. Safe action traces are written to
logs\host.log; typed content is never logged. Agent mode requires pi
authentication through pi /login or a provider API key.
Right-click the tray icon → Settings… to pick the agent model and its
reasoning effort. The list shows models with configured authentication, and
effort options adapt to the selected model. The choice persists in
%LOCALAPPDATA%\pi-os\settings.json; every new hotkey invocation uses it
(a running task keeps its own model), both switches are logged to
logs\host.log.
No environment variables are required for normal use. These optional settings change user-visible behavior:
| Variable | Meaning |
|---|---|
PI_OS_HOTKEY |
Hotkey override, e.g. Ctrl+Shift+F9 (default Ctrl+Alt+Space) |
PI_OS_INVOKE_TIMEOUT_MS |
Maximum time for each request in milliseconds (default 300000; 0 disables the timeout) |
Other PI_OS_* variables are development and test controls documented in
DEVELOPMENT.md.
Artifacts live under %LOCALAPPDATA%\pi-os\: logs\host.log,
captures\shot-*.png, plus settings.json (model + reasoning effort chosen
in the tray settings page).
| File | Read when |
|---|---|
shared/protocol/protocol.md |
You change any endpoint, port, or message shape |
shared/schemas/desktop-context.ts |
You touch the context snapshot shape (C# mirror must stay field-compatible) |
AGENTS.md |
You use a coding agent in this repository |
- Starts from one desktop shortcut and remains available in the system tray.
- Captures the active window, screenshot, focused UI element, and monitor information before showing the prompt.
- Uses a pi agent to inspect and interact with the captured window.
- Supports focusing, clicking, typing, key presses, keyboard shortcuts, and scrolling.
- Provides model and reasoning-effort settings from the tray menu.
- Supports cancellation, request timeouts, and live task status.
- Blocks computer input in protected applications such as password managers and elevated windows.
pi-os is available under the MIT License. Third-party dependencies remain subject to their own licenses.