Skip to content

vrOS Solitude v1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@justwaters justwaters released this 18 Aug 00:59
· 2 commits to main since this release
v1.0.0
43dada2

vrOS Solitude

vrOS streams your macOS desktop to an iPhone for viewing in a Cardboard-style VR headset, over a plain USB cable. This is the first release: two independent Xcode apps — a macOS Sender and an iOS Receiver — talking a small custom binary protocol over a USB-forwarded TCP connection.

How it works

  1. macOS captures the primary display with ScreenCaptureKit and hardware-encodes it to H.264 via VideoToolbox.
  2. Encoded frames are wrapped in a custom 30-byte-header binary packet and sent over TCP to localhost:2345, forwarded to the iPhone over USB via iproxy.
  3. iOS decodes the H.264 stream with VideoToolbox, uploads it to a Metal texture, and renders it in side-by-side stereo with per-eye barrel distortion and chromatic aberration correction.
  4. Head tracking and lens distortion are driven by Google's Cardboard SDK (compiled from source as a submodule, pinned at v1.34.0), so the view actually tracks your head movement in the headset rather than being a flat mirrored stream.

What's in this release

  • macOS display capture at 1920×1080 @ 30fps
  • Hardware H.264 encode → decode pipeline (VideoToolbox on both ends)
  • USB TCP transport via iproxy, custom framed binary protocol
  • Metal rendering with side-by-side stereo split
  • Barrel distortion + chromatic aberration correction for lens viewing
  • Full Cardboard SDK head tracking, including a recent fix for orientation lock, axis order, and world-lock math in landscape use
  • On-screen latency and frame-count HUD for tuning

Requirements

  • macOS 13.0+ (Ventura) for the Sender
  • iOS 16.0+ for the Receiver
  • Xcode 15.0+ (Swift 6 language mode)
  • A USB cable and a Cardboard-style (or similar) VR headset

See the README for build/setup instructions, including the required git submodule update --init --recursive for the Cardboard SDK.

Known limitations

This is a working proof-of-concept, not a polished product yet:

  • No audio capture — video only
  • No USB hot-plug handling — reconnects require relaunching
  • No key-frame request / error recovery if frames are dropped
  • Primary display only — no display-selection UI
  • No automated tests

Naming

This is the first release in the vrOS Solitude line — every v1.x.x release will ship under that codename. See CHANGELOG.md for the versioning/codename convention going forward.

Assets 2
Loading