shadow v0.5.1

Highlights

Semantic computer use

Ghost’s click and type tools now act through macOS AX, Windows UIA, and Linux AT-SPI, with each input tool returning a via field and a disposition: delivered, verified, retried, or refused. Automatic HID fallback happens only in auto mode and only for ACTION_NOT_SUPPORTED or PLATFORM_NOT_SUPPORTED with NotDelivered and known bounds; other failures return POLICY_DENIED with details.semantic_error. Shadow’s computer-use loop and AX tools use the same semantic path, and a failed step ends the run as blocked while keeping earlier steps.

Element references carry the snapshot id as @:eN and resolve strictly, so a ref from an older or different snapshot fails with STALE_REF or AMBIGUOUS_TARGET instead of matching another element; secure field values are never stored. Ghost tools return a shared GhostError with code, message, delivery, and retry information. ghost-hands enforces key policy and layout-aware keys so unsendable keys fail before posting, and macOS Cmd+A follows the active layout.

Affected packages: shadow

Changelog

shadow@0.5.1

Minor changes

• Shadow's computer use moved to an Effector trait over ghost-eyes and ghost-hands, with semantic CLICK, TYPE_TEXT and SCROLL, STALE_REF window-pid checks and blocked runs on a failed step. The AX tools became a thin adapter over ghost-eyes, ax_click's strategy parameter was removed, and mimicry undo uses the platform undo key. (#956)

Patch changes

• Updated the shadow crate lockfile dependencies. (527e01f)
• Refreshed the shadow app's Rust lockfile to pick up the upgraded and un-yanked crates. (#962)
• The computer-use agent was restructured into act, candidates, effector, input, observe, planners and verify modules, accessibility tools moved under an ax module with click, input, read and system parts, and grounding and mimicry executor code were updated. (a90a427)

Full changelog: amajorai/ryu@v0.5.0...v0.5.1

Full release notes: docs.ryuhq.com/docs/release-notes/v0.5.1