Added
-
Device twin: the host side of mirroring a physical iPhone. A
companion app on the phone (not yet shipped) will stream H.264 screen
captures overWS /devices/companion/video; the server decodes them
once per device (TwinScreen+VTH264Decoder) and hands out
Screenviews, so the existingMJPEGStream/AVCCStream
pipeline servesWS /devices/:udid/stream?format=with no changes —
the mirror is just another screen behind an existing role.
GET /devices.jsonlists connected companions. Gestures on a device
stream are rejected loudly (device control is not wired yet)
until the XCUITest control pipe lands. TheAttitudevalue type
(CoreMotion[x,y,z,w]wire order, shortest-path slerp through
quaternion sign flips, re-zero calibration) is in place for the
gyroscope-driven twin phase. NamedTwin*rather thanCompanion*
because "companion screens" already means CarPlay in this codebase.
The phone-side companion (app + ReplayKit broadcast extension +
sharedTwinWireframing) is scaffolded as a Tuist project under
Companion/DeviceTwin/. The web UI is unified rather than doubled:
/devices/:udidserves the samesim.htmlas a simulator —
target.jsis the one seam that knows the base path — with sim-only
toolbar clusters hidden and a view-only badge; connected phones
appear in a DEVICES section on the list page. The 3D stage works for
physical hardware too: model definitions gain adeviceModelsmatch
key (utsname.machineids), and when nothing matches, the browser
offers the installed models and the user's pick rides?model=on
the 3D socket — baguette still never substitutes a look-alike on its
own. The 2D page's bezel is borrowed the same way — real hardware
ships no DeviceKit chrome, sodefinition.jsontakes
?chrome=<device name>and the page offers a frame picker. And the
twin turned physical: the broadcast extension streams
CMDeviceMotionattitude overWS /devices/companion/motion
(its own socket, so video bursts never delay a pose sample) and the
3D stage follows the phone in your hand — the pose ABSOLUTE against gravity
(lay the phone down and the twin lies down; only the
compass-arbitrary heading is calibrated), rendered on a fps-paced metronome
replaying the timestamped trajectory 50 ms back (regular sampling is
what makes motion-through-video smooth; ReplayKit's own frame cadence
is irregular by design) —
the folder held simulator-list UI, and on the public surface
"devices" now means physical hardware, matching Xcode's own
Devices-vs-Simulators split.
Seedocs/features/device-twin.md. -
screenshotandinputaccept--display carplay. The two
agent-facing surfaces could not reach the CarPlay plane at all: only the
serve WebSocket read?display=carplay, and that rides the browser-trust
check. Both commands now take--display phone|carplayand bind through
the sameStreamDisplayPlanthe stream uses — enabling the external
display first, and failing closed when no framebuffer sits behind the
plane rather than quietly capturing the phone under a CarPlay label.
That refusal now says what to do about it: until the CLI gained
--display, the error only ever reached a WebSocket handler and
surfaced as its own enum dump (noMatchingPort(Baguette.DisplayKind.carPlay)),
which names the failure but not the remedy. It now names the menu that
attaches a plane, and distinguishes nothing-attached from the
attached-but-unbacked state a detach-and-re-enable leaves, which needs a
cycle through Disabled instead. Unlike the WS query, a flag value no plane
answers to is a validation error rather than a silent fallback to phone —
empty included, so--display "$PLANE"with nothing in$PLANEstops the
command instead of taking the phone behind the caller's back. Only an
absent flag means phone. Both commands reject the value before resolving
the device, so a broken command line reports itself rather than the udid.
Fixed
- Picking H.264 over a LAN address left the window black for good.
WebCodecs is secure-context-only, so a page served over plain HTTP has
noVideoDecoder— but a stored format skipped the capability probe,
and nothing stopped H.264 being stored from a browser that couldn't
play it. Every later load then threw out of the page's boot before the
toolbar and unload handler were wired, blacking out phone and CarPlay
pane alike.StreamFormatnow filters the stored preference against
what the browser can decode, the picker disables a codec it can't play,
and a decoder that won't construct falls back to MJPEG instead of
taking the page down. Fixes
#71. - Ending a CarPlay input session left the display alive and dead to
touch.IndigoHIDInput.deinitreleased the external plane's digitizer
along with the pointer service, reasoning that a digitizer outliving its
display leaves the next session a target addressing nothing. That has the
ownership backwards: the host brought that display up and is still using
it, and it outlives our process —baguette inputis one gesture long.
Removing the service left the CarPlay window on screen with nothing behind
it, unresponsive to the host's own pointer as much as to ours, and turned
target1into exactly the unregistered kind that makes
SimHIDVirtualServiceManagerthrow and takebackboarddwith it. The
digitizer now stays registered; creating is idempotent, so re-warming a
plane the host already built one for costs nothing.servewas affected
too — a closed CarPlay pane killed the window's touch — but the CLI made
it fire on every invocation.
What's Changed
- feat(cli): reach the CarPlay plane from screenshot and input by @crockalet in #70
- fix(web): never stream a format the browser has no decoder for by @crockalet in #72
- Implement core device twin functionality and enhance motion handling by @hanrw in #73
- Add tests for device matching and bundle listing in LiveDeviceModels by @hanrw in #74
Full Changelog: v0.1.96...v0.1.97