Browser-based iOS screen mirror over USB (cross-platform fork)

[renegadelink]

Hi all — I've been working on a screen-mirror subcommand that streams an iOS device's screen into your browser over USB. It's a fork of pmd3. Feedback welcome!

Repo / branch: https://github.com/renegadelink/pymobiledevice3/tree/screen-mirror

What it does

• pymobiledevice3 screen-mirror starts a small local web server; open it in any browser and you get a live view of the device.
• USB-only. No AirPlay, no jailbreak, no companion app on the device.
• Cross-platform: works on macOS, Linux, and Windows (with caveats below).

Backends (chosen automatically; you can also force one with --backend)

Backend	Platform	FPS	Notes
valeria	Linux / Windows	30–60	H.264 over USB (pip install av)
avfoundation	macOS only	30–60	CoreMediaIO; needs Camera + Screen Recording TCC (granted to your terminal app, not to Python — see below)
accessibility	any	~4	DTX fallback, always works
screenshotr	any	<1	Last-resort fallback

What it does NOT do

• No touch / remote input. This is a one-way mirror.
• No audio.

Try it

pip install -U "git+https://github.com/renegadelink/pymobiledevice3.git@screen-mirror"
pip install aiohttp av     # av only needed for the Valeria backend
pymobiledevice3 screen-mirror
# then open http://127.0.0.1:8080

macOS TCC gotcha

On macOS the avfoundation backend needs both Camera and Screen Recording permissions. macOS attributes these to the app that launched the process, not to pymobiledevice3 itself, so grant them to whichever terminal you're running the command from:

• Terminal.app, iTerm2, Ghostty, Warp, etc. → grant to that app
• VS Code / VSCodium / Cursor integrated terminal → grant to the editor app
• A tmux/screen session → grant to the terminal hosting it

Open System Settings → Privacy & Security → Camera and → Screen Recording, tick your terminal app in both, and fully quit + relaunch it (the permission only takes effect for newly-spawned processes). The startup banner will print tcc=authorized for both when it's set up correctly.

Sharing logs

If you hit issues, re-run with -v --redact and paste the log — --redact scrubs hostnames, UDIDs, AVFoundation UUIDs, and possessive device names (e.g. <Owner>'s iPad) so the output is safe to share publicly.

Feedback

• Does the auto-selected backend match what you'd expect on your OS?
• macOS: does the Camera + Screen Recording TCC flow guide you correctly the first time, or does it dead-end?
• Linux: Valeria framerate / stability on your distro and iOS version.
• Anything where the error message left you stuck.

Not opening a PR yet — want to get real-world testing in first. Thanks for taking a look.

[JJTech0130]

Not really sure what the point is behind having all these backends is: Quicktime/AVFoundation are just using Valeria under-the-hood, and I'm pretty sure it should work in almost all cases?
And I'm not sure how helpful having a screenshot backend is, since it's not exactly a very usable framerate.

In any case, what's the idea behind making it available from the browser? pymobiledevice3 is typically a CLI application, is there a reason you don't just make this a separate project that uses the public pymobiledevice3 API?

[doronz88]

I view this project as attempting to expose all device external APIs for communication. As such, I do believe adding a wrapper for the usb interaction and allowing to capture a stream using it via a nice API can be very much considered good to have within this package.
I do agree using other backends or the web UI goes against what this project aims to accomplish and should be performed as an external project.

In conclusion, as I see it, the best impl will be:

pymobiledevice3 -> USB interaction and provide API to capture frames (haven't looked yet at how this is currently performed)
some-other-package -> Wrap this to achieve web UI or anything else.

[renegadelink]

Well one important difference besides FPS between Valeria/AVFoundation and the accessibility/screenshotr backends are Valeria/AVFoundation puts the iPhone into some kind of presentation mode where the time (top left of screen) is changed to a fake time and notifications don't pop up. There maybe other side effects I'm not aware of but besides that it shows the real screen.

When using the accessibility backend, you see the device's screen entirely as it really is. I also don't really know which devices support Valeria/AVFoundation (I only have 1 device) so I thought it would be nice to have a fallback that I think should almost always work. If you guys rather it be only Valeria/AVFoundation (linux/macOS) I can take out accessibility/screenshotr no problem.

I chose the browser because it seemed like a simple way to view the screen and if you wanted to allow others over the network to see it too.

[doronz88]

Having looked at the code, we should also completely dismiss the use of pyobjc in favor of pure ctypes for the AVFoundation backend

[doronz88]

The capturing process is the python one on this case, meaning we cannot simply add our own entitlements.

[renegadelink]

ctypes over pyobjc. no problem!

[renegadelink]

Sounds good. I can pop an error if there's not a clear device to choose

[renegadelink]

I didn't try com.apple.coremedia.valeria.allow. I couldn't figure out how to make macOS give up the usb device so libusb could claim it and start sending Valeria traffic to the phone. I ended up settling for AVFoundation (QuickTime Player -> Start New Recording uses it) which also had the added benefit that the phone never has to disconnect/reconnect.

[renegadelink]

Regarding:

Known limitation: two identical-model iOS devices on the same host,
without root, cannot be disambiguated via any public CMIO/AVFoundation
API. Either run as root (Strategy 1), or expect one device to drop to
accessibility-based capture.

Honestly, it seems less than ideal to silently change methods based on factors like the number of devices plugged in. Better to be explicit and just refuse to allow picking a specific device if that really is not possible without root.

I could not figure out how to find which usb device to stream from just from a uuid and all the info pmd3 can query about that device. The only way I found was a file only root can see that can show a mapping. If someone can figure that out, that'd be great!

[doronz88]

After reviewing most of the code, I think its okay to keep the webserver as it is small enough and very much self-contained, but only for valeria and avfoundation (platform-dependent).

The high-level exposed API should be the same.

[doronz88]

Also, since these are both valeria de-facto, I would expose them both as Valeria service wrappers, just with different implementations given the platform constraints

[renegadelink]

After going back through the thread I think you're right. Kept the PR small:

• Python API for consuming the H.264 stream as it comes off the device. Two implementations behind one IOSScreenCapture interface: CoreMediaIO on macOS, libusb on Linux/Windows.
• Small CLI subcommand (pymobiledevice3 valeria) that dumps the stream to a file or stdout, so non-Python users can pipe it into ffmpeg.
• The CMIO path is pure ctypes against the public C API — no PyObjC, no Objective-C runtime.

I kept the screen-mirror webserver in pmd3 since you mentioned it was small enough, but gated it behind a [screen-mirror] extra so the default install doesn't pull aiohttp. Also rewrote it to forward H.264 straight to the browser (which decodes with hardware via WebCodecs) — no PyAV / Pillow / JPEG re-encoding on the server. Happy to extract it to a separate package if you'd rather it not live in pmd3.

[renegadelink]

pushed the new changes. let me know what you guys think!

pip install -U "git+https://github.com/renegadelink/pymobiledevice3.git@screen-mirror"
pip install aiohttp   # only needed for the browser viewer (`screen-mirror` subcommand)
  
# To a file                                                                                           
pymobiledevice3 valeria -o stream.h264 --duration 10            
ffplay stream.h264
  
# Browser viewer
pymobiledevice3 screen-mirror                                                                         
# then open http://127.0.0.1:8080 

[doronz88]

Looks good :) I'd like to review it much more in-depth, but currently this is a deatched fork. Can you please make into a PR?

[renegadelink]

done!

[renegadelink]

Okay I had to make substantial changes for linux. I closed the previous PR and made a new one.

[nanoscopic]

I will throw out there again that there is a new way to obtain full frames from devices which is by using the Debug View Hierarchy.

So far there are two non-Apple implementations existing that I'm aware of:

1. The one I wrote that I haven't released publicly yet.
2. This one: Add debugviewhierarchy: capture iOS view hierarchy via pure GDB RSP danielpaulus/go-ios#699

A python implementation of this should be created as well if the goal is to expose all the various ways to "get a video stream" in a similar fashion.

Note that the go-ios PR implementation has not yet implementation merging of the images of each section into a cohesive single image nor optimized for repeatedly doing the process rapidly which would require caching the initial steps to be able to produce a decent framerate.

One nice thing about this method is that you get the full screen without the top bar, effectively seeing underneath of that bar.