Skip to content

v0.1.0 — AirPlay + ESP32 USB HID (computer in the middle)

Latest
Latest

Choose a tag to compare

View all tags
@onehumanbeing onehumanbeing released this 14 Oct 20:51
v0.1.0
1f6a51a

First tagged release: the original architecture, before the next big redesign.

  • Eyes: the iPhone mirrors its screen over AirPlay to UxPlay on a Mac or Linux computer.
  • Hands: a ~$7 ESP32-S3 plugs into the iPhone's USB port as a mouse + keyboard (AssistiveTouch pointer), connected to the computer over Wi-Fi.
  • Brain: any OpenAI-compatible vision model (demo: Qwen3.8-Flash-Next on two DGX Sparks).
  • No Mac required, no developer account, no jailbreak.

Known limits: portrait apps only, ASCII-only typing, ~1.3 s per step.

Use this tag if you want the computer-in-the-middle setup after main moves on.

Assets 2
Loading