Skip to content

Desktop Tools 0.16.0

Choose a tag to compare

@github-actions github-actions released this 20 Oct 02:46
a9fd422

Added

  • desktop_select selects text on a PDF page in Preview. It was refused because the page's text reports itself disabled and has no selection until one is set. To copy it, activate Preview and choose Edit > Copy, since Preview disables Copy in the background. (#130)

Changed

  • Results are compact, so an agent's context fills far more slowly: the restore, focus, inspect and click loop on a TextEdit window went from 34,721 characters to 5,543. Elements list their fields once, with bounds as [x, y, width, height], real booleans and state only when set; window and app rows keep their target and drop the fields it repeats; detection metadata, timings, diagnostics, coordinate context and checked receipts are left out. A large window lists more of its 200 elements, since the limit applies to the compact result. See Result shape. DESKTOP_TOOLS_VERBOSE=1, or verbose in the client config, returns native results whole. (#144)
  • Window actions (focus, restore, minimize, maximize, full screen, move, resize) return the window's refreshed row instead of the app's whole window inventory and an inspection with a screenshot; inspect before input. (#144)
  • desktop_windows counts the untitled off-screen windows that most apps keep, such as menu bar strips, in hidden_windows instead of listing them; all: true lists them. (#144)
  • The agent's name label under the window outline is a point smaller and tighter, and edged in a darker shade of the outline color instead of white.
  • Peekaboo is vendored in native/vendor/Peekaboo, with our changes applied, instead of fetched and patched at build time. Native builds no longer fetch Peekaboo, and npm run peekaboo-test runs the vendored tests without --scratch. git diff peekaboo-upstream-4d43dc9 -- native/vendor/Peekaboo shows our changes to upstream.

Fixed

  • A point click on a sheet that Accessibility only lists inside its parent window, such as the + in System Settings > Spotlight > Search Privacy, or Cancel in the Open panel it opens, reaches the sheet. It was refused as occluded by the parent, although nothing covered the sheet. (#141)
  • A point click that finds no pressable element, or an element in a window covering the target, is refused instead of unknown, since nothing was sent, and the message no longer suggests a --foreground option the tools don't have. The occlusion refusal names the window it found. (#141)