Skip to content

Release v2.1.0

Choose a tag to compare

@github-actions github-actions released this 19 Nov 19:44
9e2f3d1

What's Changed

Added

  • axe serve --udid <UDID> keeps one process open and runs one command per standard input line, answering each with the JSON line axe --output json prints. Setup happens once, and serve keeps the input broker running between commands, so reading the screen and tapping took about 176 ms instead of 290 ms on an iPhone 17 Pro simulator (146 ms with --minimal-keys). Based on the serve mode proposed in #70 by @alex-hall and @dphurley.
  • describe-ui --minimal-keys reads only label, identifier, value, frame, role, role description, type and enabled state, about 30 ms faster on a Settings screen (#70 by @alex-hall).
  • axe device-port --udid <UDID> --port <port> saves a physical device's service directory port when AXe cannot read it on that Mac. AXe checks the port with the device first and uses it until the device's tunnel is recreated. Without --port it prints a line to paste in Terminal that finds the port with netstat and saves it (#91).
  • Agent mode can now see and use web pages. describe-ui --agent --web lists the links, buttons, text fields, checkboxes, pop-up buttons and search suggestions of the pages on screen as w refs, and tap --agent --ref w3, type --agent --focus --ref w2 and key, key-combo and key-sequence act on them, so an agent can fill in a form, clear a field, pick from a drop-down and move around a site or web app; swipe --agent on the web view's scroll area scrolls the page. Safari sheets also list their Back, Forward, Reload, Share and Open in Safari buttons and the keyboard's Previous, Next and Done; their Close button is not reachable through accessibility. It works in WKWebViews in any app, whether or not they set isInspectable, in Safari, SFSafariViewController and ASWebAuthenticationSession sign-in sheets. Only controls a tap would reach now are listed; the page text is left to screenshots. AXe finds each page with a few hit tests, walks it for controls, and checks the element at a control's point again just before the tap and before each key (when the keyboard covers a field, each key checks only that the same app is still in front). On an iPhone 18 Pro simulator (release build) a capture took about 0.29 s with --web and 0.13 s without, and agent commands without --web are as fast as before (#65).
  • axe mirror --udid <UDID> hosts one booted simulator for the free AXe Mirror app on iPhone and iPad, which shows the simulator's screen live and sends touches back. The screen is streamed as HEVC video at up to 60 frames a second with a bitrate that adapts to the link, and the viewer's touches (one or two fingers) and its Home, app switcher and Lock buttons are replayed on the simulator. The command prints one JSON line per event and runs until Ctrl+C. Connections are encrypted from end to end and go directly between the two devices when they can, through a relay when they cannot. When macOS keeps axe off the local network, the host says so (event warning, code localNetworkBlocked) and puts a viewer on the same network on the relay. It needs an Apple silicon Mac with macOS 14.5 or later. The app's source is in AXeMirror/ and can be built without the App Store. See Mirror.
  • axe mirror pair shows a one-time code, as text and as a QR code in the terminal, for the app to scan or type; it works once and for ten minutes. Only paired viewers can connect. axe mirror viewers lists them and axe mirror unpair <name or ID> removes one.
  • axe mirror --udid <UDID> --demo --demo-app <bundle ID> runs a public demo host: any viewer may connect without pairing, one at a time, and is kept inside one app, with the hardware buttons off, touches from the top and bottom edges dropped and the bitrate capped. The host checks which app is in front and, when it is another, drops the visitor's touches and brings the app back. A demo host uses its own key.
  • The source of the AXe Mirror app for iPhone and iPad is in AXeMirror/. It shares its protocol files with axe mirror.

Changed

  • AXe is now licensed under the GNU General Public License version 3 only (GPL-3.0-only) instead of the MIT License. Running axe to test an app does not put that app under the GPL; distributing AXe, or software that contains its code, must follow the GPL, and a commercial licence is available for closed-source use. Releases up to and including v2.0.4, and all source published before this change, remain available under the MIT License. The licence and third-party notices now ship in AXe's resource bundle under licenses/axe. See Licence.
  • axe type now types on the simulator's own hardware keyboard layout instead of always sending US key positions, so text arrives as written on German, French, British and other keyboards, including characters that need Option or a dead key. Before, yz@ arrived as zy“ on a German keyboard and # as £ on a British one. AXe reads the layout from the simulator's keyboard settings and the key positions from the layout data in its runtime. This covers type on every simulator route (plain, --input-mode selected, --agent), batch type steps and serve. --layout <name> names a layout and --layout us sends US key positions as before. On a physical device --layout <name> types on that layout too, with the layout data of the closest installed simulator runtime; a device does not report the keyboard in use, so it keeps US positions by default and refuses --layout auto. (#81, reported with a first implementation in #95 by @scriptease)
  • axe --output json describe-ui now takes as long as plain describe-ui, about 45 ms less on a Settings screen, with the same output.

Fixed

  • On an Apple TV simulator, AXe now fails at once with unsupportedDevice and says it supports iPhone and iPad simulators. Before, describe-ui, tap, swipe and touch waited 60 s and then said the simulator was still starting, and displays, screenshot, record-video and stream-video printed internal errors. AXe has never supported tvOS; key and button still send their input as before (#69, reported by @ashwin-raghuraman).
  • On the iPhone Duo with iOS 27.1, hinge read, hinge set, agent input and the plain screenshot no longer fail with "Motion sample reports invalid angle or velocity". That runtime reports no velocity with any hinge sample; AXe now reads it as 0 and still refuses a sample whose angle is invalid (#87, reported by @WingCH).
  • On the iPhone Duo Home screen, agent captures no longer list every icon, Dock item and status bar control twice. SpringBoard lists the windows of both screens, and AXe now leaves out the windows sized to the screen that is off, so a ref names one control. On a 27.2 Duo the agent capture took 0.22 s instead of 0.37 s folded and 0.29 s instead of 0.40 s open. Folded Home-screen icon taps that were refused as targetCovered now go through. (#87)
  • An agent tap no longer goes through when its tap point lands on bare background instead of the control. SpringBoard on the iPhone Duo can keep an icon's frame from the other screen's layout, so the frame's centre can be empty wallpaper; AXe now refuses that tap with targetCovered and sends nothing, where a copy on the screen that is off used to send a tap that did nothing. (#87)
  • Physical devices on Macs that hide network connections from AXe no longer fail with "The CoreDevice service directory port was not found" and nothing to do about it. AXe found the port with netstat, which listed no connections to an ad hoc signed AXe on two Macs with macOS 27.2.1 and an endpoint-security agent, and then with a system log scan that took 8 to 41 s and found nothing for a tunnel older than a day. AXe now reads the port with nettop in about 70 ms and no longer scans the log. When that also fails, the error prints one line to paste in Terminal that finds the port and saves it with axe device-port (#91, reported by @pbush25 and @Jonathanm10).
  • On the iPhone Duo's open screen in landscape, and on an iPad in landscape, Home-screen frames now match what is on screen, so agent taps on Dock icons, Home-screen icons and the search button go through instead of being refused as targetCovered. SpringBoard reports those frames in the screen's unrotated portrait coordinates while apps report rotated ones; AXe now rotates SpringBoard's frames. (#87)
  • On the iPhone Duo, captures no longer list Home-screen icons from the other screen. SpringBoard hands one screen's icon list the other screen's icons, with that screen's positions; AXe now asks each Home-screen button which screen it is on and leaves out the others. Those icons cannot be named on the screen where they appear, because accessibility does not list them there; tap them by coordinates from a screenshot with the capture's snapshot ID. (#87)
  • Plain describe-ui, tap --id and tap --label on the iPhone Duo now read only the active screen, as agent mode does, so each Home-screen control appears once with frames that match the screen. Plain describe-ui took 0.24 s instead of 0.35 s folded and 0.28 s instead of 0.36 s open. (#87)
  • When an agent swipe, drag or pinch is refused because AXe cannot place its start point inside the chosen area, the message no longer says the screen moved. It names the gesture, and it says to use coordinates only once a fresh screenshot shows the start point is on that area. (#87)

Installation

Homebrew

brew tap cameroncooke/axe
brew install axe

Manual

Download AXe-macOS-v2.1.0-universal.tar.gz from the assets below and extract it.

Keep the extracted payload together:

axe
Frameworks/
AXe_AXe.bundle/

Then either run ./axe from the extracted directory or symlink that axe executable onto your PATH without moving it away from Frameworks/ and AXe_AXe.bundle/.