Skip to content

v0.6.0: Computer Use in Pi's codemode

Choose a tag to compare

@tmustier tmustier released this 13 Oct 18:03
· 22 commits to main since this release
c3eb87f

Version 0.6.0 lets Pi agents use Computer Use from Pi's built-in codemode tool. The MCP interface keeps its existing 10 typed methods.

Computer Use in Pi's codemode

  • registers the 10 official methods as Pi tools named computer_use_<method>, such as computer_use_get_app_state and computer_use_click
  • gives them codemode exposure in a computer_use namespace, so agents combine Computer Use calls with other Pi and MCP tools in one script
  • computer_use_get_app_state resolves to { app, text, screenshot }, and image(state.screenshot) shows the screenshot
  • a failed call rejects with the official error text
  • calls after get_app_state reuse one signed Computer Use session until the agent settles
const state = await tools.computer_use_get_app_state({ app: "TextEdit" });
text(state.text);
await tools.computer_use_click({ app: "TextEdit", element_index: "7" });
await tools.computer_use_type_text({ app: "TextEdit", text: "hello" });

The extension does not change which tools are active. Turn on codemode with "defaultTools": ["+codemode"] in settings.json or pi --tools +codemode. If neither codemode nor a Computer Use tool is active, the extension shows one notice at the first prompt.

Dependencies

Upgrade notes

This is a breaking change for Pi:

  • The computer_use({ code }) tool and its sky object are removed. Call the methods from codemode instead.
  • Pi 0.99.0 or newer is required.
  • An explicit --tools list must include the tools you want. Use --tools +codemode to add codemode to the defaults.

The MCP interface is unchanged.

pi install npm:codex-computer-use-mcp@0.6.0

Validation

Strict Oxlint, core and Pi type checks, and 54 tests pass, including a packed-package install and a live call through the signed official broker. A live Pi run used openai-codex/gpt-6.1-sol with Pi's built-in codemode. The model's scripts combined Computer Use calls with read to edit and save a TextEdit file, and the file on disk was checked afterwards. A live MCP stdio list_apps call passed on SDK 1.32.1. A fresh npm install reports no production vulnerabilities.

Compare v0.5.2 to v0.6.0