v0.15.0

Added

• iOS: Xcode 27 Resize Mode (a "resizable app session", started from Device Hub or xcrun devicectl device appResize start) is now detected, and every affected verb is honest about it instead of pretending to work. During a session the frontmost app renders on a virtual Resizable display while the main display shows only the wallpaper; every simulator primitive sim-use drives — HID touch, the framebuffer, the accessibility point hit-test — is bound to the main display. ui now carries a resizable_app_session advisory ([i] … in text, top-level advisory in --json) naming the scene size and the device size; screenshot captures the Resizable display (the app, at the scene's size) and says so in the same advisory; tap, long-press, swipe, touch, gesture, multi-touch, paste --via-menu (a long-press plus menu taps) and a batch containing a touch step fail with an explanation and a hint instead of reporting a tap that was never delivered; a batch re-checks before every touch step, so a session that begins mid-batch, or inside the cache window, stops the batch at its first touch step with nothing sent. record-video and stream-video fail the same way, before creating a file or emitting a byte — both capture only the main display, and the hint gives the xcrun simctl io <udid> recordVideo --display=<uuid> form that records the Resizable one. type, Cmd+V paste and the key verbs are untouched — keyboard HID does reach the resized app. The check is one xcrun devicectl device info appResize per device per 5 s (~0.3 s, cached in the daemon) — re-run early when ui or a selector tap sees a scene size that is neither the device's own size nor its landscape swap and that it has not probed before, so a session that just started is caught at once while a display whose accessibility size never matches its framebuffer (iPhone mini) pays no more than anyone else; only a scene at exactly the device's own size, and standalone raw -x/-y taps, wait out the window. The check runs on every toolchain whose devicectl knows the query (Xcode 26.6 does), since another Xcode or Device Hub may own the session; SIM_USE_RESIZE_SESSION_CHECK=0 disables it (the per-device daemon reads it at spawn). The bundled skill's preflight now requires sim-use 0.15.0 or newer, because its Resize Mode guidance relies on this detection. Real touch routing to the Resizable display is a separate, larger piece of work (upstream idb's multi-display plumbing plus the scene geometry) — see docs/ai/xxxx-resizable-app-session/README.md. (#143, #150 — thanks @iXerol!)
• Linux build, Android only: Package.swift now declares an #if os(Linux) target graph (SimUseCore, AndroidBackend, and a new SimUseLinux executable) that builds with a Swift 6 toolchain and drives Android devices exactly as the macOS build does — ui, tap, long-press, type, swipe, screenshot, app-state, the android namespace and the per-device daemon. long-press and app-state share their Android implementation with the macOS commands, so flags and JSON match. iOS verbs, video capture, the Viewer and init stay macOS-only; on Linux, invoking one prints an error naming the verb instead of a misleading parse error. The macOS manifest is unchanged. scripts/install-linux.sh builds and installs it; see docs/linux.md. (#141, #147 — thanks @iopass4!)
• Remote adb servers: when ADB_SERVER_SOCKET points at another machine (tcp:<host>:<port>, e.g. WSL using the Windows host's adb), or ANDROID_ADB_SERVER_ADDRESS does when no socket is set, the bridge is reached on that host — where adb forward actually listens — instead of 127.0.0.1. SIM_USE_BRIDGE_HOST overrides the host. The cached forward and bridge token are scoped to that connection: switching adb servers restarts the device's daemon and re-creates them, and a session restored from disk is reused only while adb forward --list still lists its forward, so a restored token is not sent to another host or to an unrelated listener on the old port. A session that a running daemon already holds is not rechecked before each request, so a forward that another tool changes under it goes unnoticed until a request fails. Each daemon request also carries the client's connection, and a daemon started for another connection refuses the request without running it; this covers a daemon that is too busy with another client's command to answer the restart check in time. This covers every adb serial, including wireless-debugging serials too long for the Android UDID heuristic, and a failed adb forward --list is reported instead of opening another forward and leaving the old one registered. Equivalent spellings of the default server (no variables, ANDROID_ADB_SERVER_PORT=5037, tcp:localhost:5037) count as one connection, so switching between them neither restarts the daemon nor leaves a forward behind; when only the bridge host changes, the old forward is removed before a new one is opened. A forward on a different adb server is left in place (see docs/linux.md). android init reports the host it reached the bridge on as http_endpoint (and bridgeHost in --json) instead of always printing localhost. (#141, #147 — thanks @iopass4!)
• sim-use android describe-ui accepts ui, matching the top-level verb. (#141, #147 — thanks @iopass4!)
• tap accepts --coordinate-space for explicit -x/-y / --point coordinates, matching swipe and touch. ui transforms outline (visual-space) coordinates through the same orientation calibration selector taps already ride, so a coordinate read off describe-ui lands where it was read on a rotated device; native (device-native portrait, the default) is unchanged and stays zero-cost. Aliases and selectors are already resolved in ui space and reject the flag rather than silently ignoring it. Batch tap steps ride the batch-wide calibration, and android tap accepts the flag for parity and ignores it. The bundled skill's Act table and pitfalls row now name tap alongside swipe/touch. (#142, #144 — thanks @iXerol!)
• stream-video --format h264 now works on iOS, not just Android: a native FBVideoStream H.264 stream, carried in MPEG-TS, copied straight to stdout with no host-side codec pass. On a booted iPhone 17 Pro this delivers ~24 fps at ~1.3 MB per 6 s, against the screenshot loop's ~4.1 fps at ~11 MB — roughly 5.9x the frame rate for an eighth of the bytes. Preview it with sim-use stream-video --format h264 --device $UDID | ffplay -f mpegts -probesize 32768 -i -. It streams at a constant --fps, default 30 (the screenshot formats keep their default of 10). MPEG-TS rather than the Android leg's bare Annex B on purpose: Annex B carries no presentation timestamps, so a player paces on a guessed frame rate (ffprobe reads a bare stream as 25 fps regardless of --fps) and a 30 fps capture drifts ~5 frames further behind every second, unbounded — minutes of lag within a few minutes of viewing. MPEG-TS carries PTS/DTS, and the player's queue measured empty across a 60 s run. The top-level stream-video no longer rejects h264 for iOS targets, so the only platform-exclusive format left is the experimental iOS-only bgra. (#132, #133)

Changed

• iOS stream-video --format mjpeg / raw / ffmpeg are deprecated and now warn once on stderr, pointing at --format h264. On a simulator the screenshot loop has no remaining advantage — h264 measured 144 frames in 1.33 MB against mjpeg's 24 in 11.2 MB over the same 6 s — and it is available on every booted simulator, so these will be removed in a later release. Android keeps its equivalents and does not warn: adb screenrecord is unavailable on some devices, and there the screencap loop is the only way to stream at all. bgra is unaffected (raw unencoded pixels are not something H.264 substitutes for). (#134, #138)

• Android stream-video --format h264 is now carried in MPEG-TS instead of the bare Annex B screenrecord produces, matching the iOS surface — one ffplay -f mpegts -probesize 32768 -i - works on both platforms. An elementary stream carries no presentation timestamps, so a player guesses the frame rate (ffprobe reads it as 25 fps regardless of the real rate) and, when the device encodes faster, drains slower than sim-use fills: the lag grew without bound instead of settling, measured at 578 KB of queued video after 20 s and 1376 KB after 30 s of interaction, and observed in practice as a picture minutes behind the device. Frames are re-containered, never re-encoded — a new MPEGTSMuxer wraps them with a PTS taken from host arrival time (screenrecord is genuinely variable-frame-rate, so wall-clock arrival is the honest timeline). The player's queue now measures empty across a 30 s run. Program tables are emitted up front and re-announced every 100 ms, so a consumer attaching mid-stream can start decoding and a closed pipe is noticed even on a still screen; each picture carries its own clock reference, and a picture resuming after an idle stretch declares a transport discontinuity, since the timeline skipped real elapsed time rather than advancing through it; --format h264 now reports a frame count in its summary because re-containering parses the stream. End-to-end latency measures ~0.6 s on an emulator, essentially all of it screenrecord's own encode delay (0.58 s measured with sim-use out of the path).

  The stream's timeline advances by the real inter-frame gap but never by more than 0.2 s, which deliberately makes it not a reproduction of the wall clock. Capture is variable-frame-rate, so a still screen produces no frames; stamping the next picture with its true arrival time would open a hole in the timeline as long as the pause, and a player has to play through that hole before reaching the new picture — seen as a preview running minutes behind after the screen had been idle. Measured after the fix, time for the picture to catch up once interaction resumes: 0.45 s after 10 s idle, 0.51 s after 30 s, 1.20 s after 60 s; previously this tracked the idle duration. Reproducing the wall clock remains record-video's job, and it keeps real arrival times for exactly that reason. (#135, #136)

• --format h264's documented preview command is ffplay -f mpegts -probesize 32768 -i -. The previous -probesize 32 -fflags nobuffer never opened an Android stream: 32 bytes cannot identify the codec, and -fflags nobuffer starves the probe. -probesize 32768 bounds the open by bytes, which is what matters on a variable-frame-rate source whose still screen sends only program tables — ffplay's default budget is measured in seconds of media and can take arbitrarily long in wall-clock terms to fill. -analyzeduration 0 is deliberately not part of the command: zero selects ffmpeg's default analysis window rather than disabling it, and the open measured identically with and without it. A still Android screen produces no pictures at all, so the player opens once the device moves; iOS is constant-frame-rate and opens at once. (#136)

• iOS record-video and stream-video now share one set of H.264 encoder settings, built by FBVideoStreamConfiguration.h264Capture(fps:quality:scale:transport:), so the two verbs cannot drift apart on how the picture is encoded. They still differ downstream in transport and sink — record-video into the shared H264MuxingPipeline → MP4 (the muxer Android has always used, and which was written for both), stream-video into stdout. Recording no longer goes through idb's separate in-process file writer. Output is equivalent: the MP4 stays a regular non-fragmented file that QuickTime, Finder and GIFTranscoder all read, --fps is still honoured as a constant rate, and the finalize path still survives a 100 ms-grace SIGTERM across repeated runs. Frame timestamps now come from host arrival time rather than the encoder's sample clock, which measures as ~6 ms of inter-frame jitter against the previous ~3 ms — both far inside a 33 ms frame at 30 fps. (#132, #133)

Fixed

• iOS: with Device Hub open (Xcode 27), batch --stdin, batch --file, type --stdin and paste --stdin reported success but delivered nothing. These modes ran the command in the CLI process instead of the per-device daemon, because the daemon cannot read the caller's stdin. With Device Hub open, HID input goes through CoreDevice's dtuhidd, which takes about 0.4 s to set up each new connection before it reads any event. The CLI process exited first, and dtuhidd dropped the queued taps and keys. The client now reads the input and forwards it to the daemon inline, as --step arguments or as the text, so every event comes from the daemon's long-lived connection. For type --file and paste --file, a relative path now resolves against the caller's directory. Before, the path went to the daemon, which resolved it against its own directory. SIM_USE_NO_DAEMON=1 still runs in-process and can drop input with Device Hub open. (#154)
• iOS: during an Xcode 27 resizable app session, ui no longer splices SpringBoard elements (home-screen widgets, the Dynamic Island, a 402x874 Group) into the app's tree, and no longer reports Screen orientation could not be confirmed … — or a confident (landscape-right) on an upright device — for a scene whose size merely differs from the device's. Both came from the accessibility point hit-test resolving on the main display, where SpringBoard is frontmost, while the tree fetch follows the app: the collapsed-children recovery walk adopted those hits as children, and the orientation calibrator read them as probe evidence. Under a session the tree is now returned as fetched, in identity calibration, with the resizable_app_session advisory explaining why. ui --point carries the same advisory, since its answer describes SpringBoard rather than the app. (#143, #150 — thanks @iXerol!)
• Android: a command that cannot reach the bridge even after reconnecting (wrong SIM_USE_BRIDGE_HOST, a firewall, an un-initialised device) now removes the forward it opened before reporting the error. No session had been saved for that forward, so nothing ever removed it and each failing attempt left one more forward on the adb server. (#147)
• Android: dropping a stale bridge forward now names the device (adb -s <serial> forward --remove). With more than one device on the adb server, the old command was rejected with "more than one device/emulator" and the error was swallowed, so every reconnect left another forward behind. (#147)
• iOS stream-video --format h264 / bgra: a consumer that stops reading (ffplay paused, a wedged downstream tool) no longer makes Ctrl-C hang. The native stream went through idb's blocking file writer, which sat in write(2) on the encoder thread with stopStreaming() waiting behind it, so the command could not end short of SIGKILL. It now writes through the same interruptible stdout sink as Android — one that waits for room and checks cancellation between waits, and leaves the descriptor's flags untouched so the terminal (or stderr under 2>&1) is never switched into non-blocking mode. A consumer that closes the pipe now ends the stream in an orderly way (stop messages, exit 0) instead of the process dying of SIGPIPE. (#138)
• The bundled skill preflight now reports a content warning instead of an unqualified pass when ui succeeds with a remote_content_recovery advisory or with an empty outline on a simulator, emulator or Android device. These reads still exit successfully. The warning points at a new Missing app controls in the outline pitfall, which covers the iOS simulator ApplicationAccessibilityEnabled launch-time setting. (#139, #145 — thanks @SunsetWan!)
• The bundled skill preflight now rejects an older sim-use CLI before device discovery and prints the Homebrew upgrade command, instead of misdiagnosing newly documented device types as disconnected. (#127 — thanks @BoWuGit!)
• iOS simulator selector-ambiguity errors now give advice an agent can act on. The multipleMatches hint de-duplicated labels, so three matches that share one label were reported as matches (1); it now lists one 'label' [Type] #id entry per match — the shape the physical-device backend already uses — and the count is the real number of matches, also past the 10-entry display cap. An ambiguous --id no longer recommends --id, and every ambiguity message now names --element-type / --frame to narrow the match. The describe-ui recovery tip and the missing outline-cache error recommend --device instead of the deprecated --udid. (#146)