Skip to content

v0.3.16

Choose a tag to compare

@github-actions github-actions released this 03 Aug 04:21
· 60 commits to main since this release

Three classes of on-screen element that snapshots could never see are now captured, and the accessibility layer got cheaper along the way.

Highlights

Menu bar extras from background apps. Status icons for tools like Docker, Ollama, LM Studio and Claude were invisible to every snapshot. They belong to their own process, which is neither frontmost nor system UI, so nothing ever asked them.

Notification Centre, including the banners. Owned by com.apple.notificationcenterui, which was not scanned at all — so Clear All, Edit Widgets and the notifications themselves were all missing.

Cheaper phase-2 fetches. Elements are no longer asked for attributes their role cannot carry. Control Centre's scan went from 25.5 ms to 19.4 ms.

Added

  • Menu bar extras owned by background applications. Apps owning a non-empty AXExtrasMenuBar are discovered and scanned — only their extras menu bar, since walking a full window tree would cost far more than the single node it contributes. Discovery is cached against the set of running process ids, so a repeat capture pays 0.15 ms rather than 7.36 ms.
  • Names for unnamed extras. These are pure icons with no title, description, value or identifier, so name-based filtering removed them despite their being plainly visible. An unnamed AXMenuBarItem now takes the owning application's name. Only unnamed items are affected, so an app's real menus keep their own titles.
  • Notification Centre contents, including notification banners. A banner is an AXGroup carrying the subrole AXNotificationCenterBanner — not an interactive role — so a new INTERACTIVE_SUBROLES set makes it interactive regardless of role, and Subrole moved into the phase-1 attribute batch since interactivity now depends on it.

Changed

  • Phase 2 asks only for what a role can carry. An attribute an element cannot have still costs work: the application has to receive the request, look, and answer "unsupported". Against Control Centre's status items, five of eleven were never answered, and dropping them halved that fetch (6.88 ms → 3.22 ms for 11 elements). Which attributes can be answered is a property of the role rather than of the application — a menu bar item structurally has no URL, a link has no text selection.

Notes

Control Centre's six disabled extras remain excluded. They sit at (0, 900) with zero size and are filtered on geometry, so naming unnamed extras after their app does not resurrect them.

One approach was measured and rejected. Treating "advertises an AXPress action" as an interactivity signal looked like the general fix for notification banners, and INTERACTIVE_ACTIONS already existed in the configuration unused. It added 118 nodes to a 95-node capture — 66 AXGroup, 40 AXStaticText, several named with icon-font glyphs — because Electron applications attach press actions to wrapper elements liberally. Matching on the specific subrole is surgical by comparison.

Capture cost rises by roughly 4 ms for Notification Centre and 6.5 ms for the menu bar extras, partly offset by the phase-2 change. Both buy elements that are on screen and were previously unreachable.

The screenshot path is unchanged in this release.

Full Changelog: v0.3.15...v0.3.16