Skip to content

v0.14.0

Latest

Choose a tag to compare

@onevcat onevcat released this 27 Aug 10:20
· 1 commit to main since this release
3a2f648

Added

  • Physical iPhones and iPads are now first-class targets of the top-level verbs: sim-use ui, sim-use tap (#<id> / --id / --label / --label-contains / --element-type forms) and sim-use screenshot route a physical UDID to the audit/CoreDevice backends automatically, instead of rejecting it with a pointer at the ios-device namespace. Every other verb and form (coordinates, @N/#N aliases, swipe/gesture/touch/multi-touch/long-press, type/paste, button, keyboard-state, app-state, recording, --value/--label-regex/--frame/--duration/--wait-timeout) rejects per verb with a TargetCapabilityError stating why the channel cannot carry it and the nearest alternative (the hint in --json) — Platform.iOSDevice joins PlatformRouter, so the compiler enumerates every top-level capability decision — 15 device-scoped verbs in all (3 routed, 12 rejected). Routed physical results state their restricted shape explicitly: describe-ui carries "kind":"physical" with no raw/entries/lists/screen (the outline text, #ids included, is the payload), and tap carries "kind":"physical" with the matched element instead of x/y. Physical targets always run in-process — no per-UDID daemon is spawned for them (daemon session persistence is tracked separately). The ios-device namespace stays as the physical-only peer of ios/android (ECID addressing, tree-tuning flags); sim-use ios <verb> remains simulator-only and still fast-fails on a physical UDID, now pointing at the routed top-level verbs. The docs gained an explicit verb × target capability matrix. (#126)

  • sim-use devices now covers every target: USB-attached physical iPhones/iPads are listed alongside simulators and Android devices (FBDeviceControl discovery, ~0.4 s with a device attached / ~1 s with none, run in parallel with the simctl and adb listers), and every row carries a new kindsimulator / emulator / physical — orthogonal to platform, as a KIND table column and an additive kind key in --json (older payloads without the key decode with an inferred kind). Android's emulator-vs-physical distinction is now stated instead of being inferable only from the serial shape. A freshly attached iPhone may be listed by ECID until a session opens; the ios-device verbs accept either identifier. --no-physical-ios opts out of the FBDeviceControl side entirely; the Viewer passes it (its /api/devices also forwards kind now), since its UI is built on coordinate taps, frames and video streaming — capabilities the physical channel doesn't carry (the routed ui/tap/screenshot subset isn't enough to drive the SPA). (#122)

  • sim-use ios-device (experimental): drive a development-signed app on a physical iPhone or iPad. devices lists attached devices, ui prints the foreground app's accessibility tree, and tap --label / --label-contains sends Activate to one unambiguous match. sim-use installs and signs no runner and needs no Developer Disk Image; the device must be unlocked and the target app must have get-task-allow=true. This channel intentionally omits coordinate tap, swipe and gesture because the daemon exposes no element geometry. (#93, #99 — thanks @subdiox!)

  • sim-use ios-device ui now renders each element's accessibility identifier as #id, and sim-use ios-device tap accepts it as a positional #<id> or --id (mirroring the simulator tap). This is a stable handle to prefer when a label is dynamic — a navigation-bar back button is labelled with the previous screen's title but keeps #BackButton. The @N alias and coordinate forms remain unavailable on this channel (handles expire between processes; the daemon exposes no geometry). Label and identifier matching go through the same case-sensitive SelectorTextMatcher policy the simulator and Android surfaces already use, so a selector behaves identically across all three. (#114)

  • sim-use ios-device verbs now support --json, emitting the same {ok, data} envelope as the simulator and Android surfaces so the agent-side parser stays shared: devices returns unified device rows (the deviceId / kind / runtime schema of top-level sim-use devices --json), ui returns the outline text plus structured rows (depth / role / label / optional identifier) with element/node counts and timing, tap returns the matched element (action / role / label / optional identifier, the id trimmed exactly as the outline shows it), and screenshot returns the saved path. The ui schema deliberately carries no @N aliases — element handles expire between processes. Structurally the four verbs adopted SimUseExecutableCommand (the shared execute()/format() split); they still run fully in-process — daemon session persistence is tracked separately (#120). Error output moved its recovery advice to the shared hint channel: the --json failure envelope carries a machine-readable hint field, and the text path prints the same advice as a Hint: line after the error instead of embedding it in the message. (#123)

  • sim-use ios-device screenshot: capture a PNG of a connected iPhone or iPad display. Capture runs over CoreDevice (xcrun devicectl device capture screenshot) rather than the accessibility audit channel, so — unlike ui and tap — it is not limited to development-signed foreground apps: whatever is on screen is captured, SpringBoard and system apps included. Device selection matches the other ios-device verbs (--device optional with exactly one attached), and --output follows the shared path semantics with a Device Screenshot - <device name> - <timestamp>.png default. A rejected path or a capture that fails mid-flight never removes an existing file at --output: the image lands in a temporary sibling and replaces the target only on success. The --output path resolution shared by the simulator screenshot and the video verbs is now factored into one OutputFilePath helper instead of two per-target copies. (#118)

Fixed

  • screenshot / ios screenshot: a simulator name containing path separators (simctl accepts any free text, e.g. My iPhone/Work) no longer turns the default output filename into a directory hierarchy — the name is collapsed into a single path component, matching ios-device screenshot. The Android default filename embeds the adb serial, whose accepted charset already excludes separators; it is now sanitised too as defence in depth. Video default filenames embed no user-controlled text and were already safe. (#119)
  • sim-use ios-device ui no longer drops the navigation-bar back button (and any other element whose token the daemon aliases with the root). On a pushed screen deviceFetchSpecialElement: 0 returns the back button as the root, so seeding the walk's visited set with the raw root token silently discarded it; the walk now dedups on (token, summary, role), so the back button appears in the outline and is tappable with the ordinary tap. (#114)
  • sim-use ios-device discovery bails in ~1 s when no iPhone is attached, instead of waiting the full 5 s timeout. An empty attachment set never satisfies the quiescence rule (it needs a non-empty, unchanged set), so the discovery loop used to run to the deadline on every device-less host; it now gives up after a short grace once nothing has appeared. A device that is present still settles in ~0.4 s, and a multi-device attach burst still coalesces (the grace only applies until the first device is seen). (#117)
  • sim-use ios (simulator-only) verbs reject a physical iOS device UDID at resolution time — pointing at the routed top-level verbs — instead of misclassifying it as an Android serial and diagnosing a plugged-in iPhone as "not reachable via adb". (#99, #126)
  • Wait for physical-device attachment notifications to settle so multiple USB-connected iOS devices are all discovered. (#99)
  • Reject empty physical-device hierarchies, missing or ambiguous tap targets, and invalid hierarchy concurrency instead of reporting success or silently choosing an element. (#99)
  • Correlate DTX replies by both identifier and conversation index so unsolicited device events cannot satisfy an unrelated pending request. (#99)
  • record-video --gif-markers (all three surfaces): bracket a GIF with START/END marker cards (~1 s each) so the forever-looping clip has a visible boundary. Opt-in — the default output remains a faithful capture of the screen. A failed card render degrades to a marker-less GIF instead of failing the transcode. (#100, #101 — thanks @kws0210!)