Verify usbmux runner transport on multi-device and unpaired/untrusted hardware

[thymikee]

Follow-up to #1403, which shipped in #1517. Two of that issue's experiments needed hardware that was not available at the time, so they were never run. Everything below is unverified rather than known-broken.

Multi-attached-device selection

#1403 asked to "exercise multiple attached devices and confirm UDID-to-device selection is unambiguous". Only one device was ever connected.

Structurally this looks sound: usbmuxd's SerialNumber is exactly the dashed hardware UDID that DeviceInfo.id already carries for devicectl-listed devices, and readDeviceIdFromList matches on it, with usbmuxd's own DeviceID used only as the mux handle. So selection should be exact rather than positional. That is a code reading, not evidence.

Worth checking with two or more cabled iPhones:

• --device/--udid selection routes each command to the intended device
• two concurrent sessions on different devices do not cross transports
• a device attached or detached mid-session changes only its own routing
• ListDevices ordering has no influence on which device is picked

Unpaired / untrusted / unavailable devices

#1403 asked to "verify failure behavior for locked, disconnected, unpaired, and unavailable devices". Locked and disconnected are covered:

• locked is transport-independent — the runner wedges identically on both routes (SBMainWorkspace … Locked, xcodebuild Unlock <device> to Continue)
• disconnected is the terminal verdict added in feat(ios): reach physical devices through usbmux first, tunnel as fallback #1517, with a regression test

Unpaired, untrusted, and unavailable were never exercised. The open question is whether usbmuxd distinguishes "attached but not trusted" from "not attached" clearly enough for the route resolver to tell them apart. Today anything that is not a successful lookup surfaces as usbmuxDeviceAttached: false, which on a CoreDevice device triggers a tunnel fallback that will also fail — so the user may end up with a tunnel-shaped error for what is really a trust problem, instead of the "trust this Mac" hint.

Wanted: the observed usbmuxd behaviour for each state, and if they are distinguishable, a mapping to the right hint.

Notes for whoever runs this

• bin/agent-device.mjs runs dist/, so pnpm build first or you measure stale code
• per-request diagnostics only record exec/devicectl phases under --debug; their absence without it is not evidence
• assert real command output, not just wall-clock — a failing command also returns fast
• set Auto-Lock to Never, or the runner dies mid-series

[thymikee]

Partly covered by #1523 (merged). Narrowing this issue to what genuinely still needs hardware.

Done in #1523

UDID→device selection is no longer just a code reading. Hermetic coverage against the fake usbmuxd now asserts that selection follows the UDID rather than the position in ListDevices — with the target first, last, and in the middle, checking the DeviceID actually sent in Connect — and that a device whose UDID merely shares a prefix with the requested one never matches, with no Connect attempted.

The "unavailable device" leg turned up a real bug rather than a confirmation. usbmuxd's Connect result codes were probed against the real daemon (unknown DeviceID → 2, closed port on a live device → 3) and every non-zero code had been collapsing into one error with a cable hint. Result 2 now raises the unattached verdict, so a device that disappears between ListDevices and Connect falls back to its CoreDevice tunnel instead of failing with a cable hint while Wi-Fi works. Result 3 now reports that the runner port is not bound yet.

That also resolves the concern raised in this issue's original description: an attached-but-problematic device does not collapse into "not attached" — ListDevices and Connect failures are now distinguishable.

Still open, still needs hardware

1. Two or more cabled devices, concurrently. The selection logic is covered; what is not is the live behaviour — two sessions driving two devices at once without crossing transports, and attach/detach of one device not disturbing the other's routing.
2. Unpaired / untrusted devices. Never exercised. Open question: does an attached-but-untrusted device appear in ListDevices and fail at Connect (in which case it maps onto the result-code handling from fix(ios): map usbmux connect result codes to the right verdict #1523), or is it absent entirely (in which case it is indistinguishable from unplugged and the "trust this Mac" hint is the only thing carrying that meaning)? Answering that needs a device that has not trusted the host, so it should not be done by unpairing someone's working device casually.

Both are verification rather than known defects.

[thymikee]

Ran both legs live on a two-device host (thymikee-iphone + a second iPhone 13 that had never been used for development).

Multi-device selection — verified, no Developer Mode needed

Both devices appear in usbmuxd with distinct handles: DeviceID=14 → 00008150-…, DeviceID=15 → 00008110-….

Selection was proven by exploiting an asymmetry instead of needing both devices runnable: one phone had a live runner on port 64840, the other had nothing there. Driving the repo's own client at the same port with three different UDIDs:

target (port 64840)	result
00008150-… (runner is here)	HTTP 200, real runner payload
00008110-… (no runner)	usbmuxResult=3, "runner is not listening on the device port"
bogus UDID	DEVICE_NOT_FOUND

Same port, three UDIDs, three outcomes. Routing follows the UDID, not list position or "first attached device", and a bogus UDID is rejected rather than silently falling through to an attached phone. This is the live counterpart of the hermetic coverage in #1523, and it also confirms that mapping behaves correctly with two devices present.

Concurrent sessions do not interfere: two opens ran simultaneously against separate daemons; one succeeded while the other failed for its own unrelated reason, with no lease contention or crossed transports.

Unpaired / untrusted — answered, and it refutes this issue's premise

I worried an untrusted device would collapse into "not attached" and lose the trust hint. It does not:

• an unpaired device does appear in usbmuxd ListDevices (so it is distinguishable from unplugged at the transport layer)
• its Connect returns 3, identical to a paired device with no runner — so usbmux alone cannot separate "unpaired" from "runner not started"
• that never bites, because the launch path fails first, in ~5 s, with The device must be paired before it can be connected

So the transport failure semantics are fine. What is not fine is what the user is told, which is now #1527:

• a device that is unlocked, trusted and available (paired) still fails with a disk-image mount error while Developer Mode is off, and the hint told the user to check trust and Xcode — all already true
• tapping Trust without entering the device passcode leaves the device unpaired; several retries reported "unpaired" until pairing was completed properly

Also worth noting separately: discovery advertises an unpaired device as fully usable. devices --json returns it as booted: true with nothing marking it unavailable, even though xcdevice reports "not available because it is unpaired". An agent selecting by name would pick it and only learn on open.

Left over

Only attach/detach isolation while sessions are live, which needs someone physically unplugging a cable mid-run. Everything else in this issue is now covered.

[thymikee]

Attach/detach isolation — verified live, closing the last leg

Two phones cabled, an active session on thymikee-iphone, cables pulled by hand.

Detaching the other device does nothing to a live session. With the iPhone 13 unplugged mid-session, thymikee-iphone kept serving snapshots at 0.45 s — byte-identical behaviour to before the unplug — and a tap still landed. No crossed routing, no lease disturbance.

Detaching the device under test degrades to the tunnel instead of breaking. Pulling thymikee-iphone's own cable while its session was live:

state	devicectl tunnel probes	snapshot
cabled	0	~0.45 s
unplugged mid-session	5	5.6 s — session survives
replugged	0	3.7 s recovering, then 0.34 s

The session never broke. It moved from usbmux to the CoreDevice tunnel when the cable went, and back to usbmux when it returned. This is the mid-session transition #1517's fallback was designed for, exercised as a real transition rather than a cable-out-from-start run.

Incidental but relevant: usbmuxd hands out new DeviceIDs on re-attach — the same two phones came back as 17/16 having been 14/15 — while their UDIDs are stable. That is exactly why routing has to key on the UDID, and it is now demonstrated rather than assumed.

Possible follow-up, not a blocker: the unplugged command spent 5 devicectl device info details spawns (~3.8 s of its 5.6 s). Each connect retry re-resolves the tunnel rather than reusing what the previous attempt just learned. That is tunnel-route behaviour predating #1517, though usbmux-first now adds a failed usbmux attempt ahead of each.

Status

Everything in this issue is now verified live except two fully operational sessions driving two devices simultaneously. Selection by UDID, concurrent opens not contending, unpaired behaviour, and attach/detach isolation are all covered. The remaining item is blocked only on Developer Mode being enabled on the second device.

[thymikee]

Final status — recommending this be closed

Everything in this issue has been verified on real hardware except one item, which is blocked by hardware that cannot be obtained: the second iPhone cannot be added to the signing account, so the XCTest runner cannot be installed on it, so it can never host a live session.

Verified live:

• Selection follows the UDID. Three-way discrimination on one port with two phones attached: the phone hosting the runner answered HTTP 200, the other answered usbmuxResult=3, a bogus UDID answered DEVICE_NOT_FOUND. Not position, not "first attached device".
• usbmuxd reassigns DeviceIDs across re-attach (the same phones returned as 17/16 having been 14/15) while UDIDs stayed stable — which is precisely why routing must key on the UDID, now demonstrated rather than assumed.
• Concurrent sessions do not contend. Two opens against two devices on separate daemons both succeeded; each device's failures stayed its own.
• Attach/detach isolation. Unplugging the other device left a live session byte-identical at 0.45 s. Unplugging the device under test degraded it to the CoreDevice tunnel (5 probes, 5.6 s) without breaking the session, and replugging returned it to usbmux (0 probes, 0.34 s).
• Unpaired devices appear in usbmuxd and answer Connect with 3 exactly like any port-closed device, so usbmux alone cannot separate "unpaired" from "runner not started" — but that never bites, because the launch path fails first in ~5 s with an accurate pairing error.

Not verified, and not obtainable here: two fully operational sessions driving two devices simultaneously. Everything up to the runner was exercised on both devices concurrently; only the runner-backed commands on the second device were unreachable.

What the exercise actually produced was two messaging fixes rather than transport defects — the transport behaved correctly throughout:

• fix(ios): name Developer Mode and pairing as the real launch blockers #1527 (merged): a device that is unlocked, trusted and available (paired) was told to check trust and Xcode when the real blocker was Developer Mode being off; pairing advice now also mentions the passcode step, without which tapping Trust silently leaves the device unpaired.
• fix(ios): diagnose a runner that cannot install, instead of blaming the screen #1529: a runner that could not install for provisioning reasons was reported as "the current screen is overwhelming the accessibility capture", with screenshot suggested as the workaround — which needs the same runner and fails identically.

I would close this. The transport questions it was opened to answer are all answered, and the one remaining item is an account/hardware constraint rather than an open engineering question.