Skip to content

mondain/moq-rn

BranchesTags

Repository files navigation

MOQ React Native

Two React Native apps for the MOQ (Media over QUIC) protocol: a publisher (camera/mic capture and publish) and a subscriber (subscribe and playback). Built as a monorepo with a shared Rust core.

Architecture

apps/
  publisher/     - React Native publisher app
  subscriber/    - React Native subscriber/player app
packages/
  moq-core/      - Rust crate: QUIC transport, MOQ protocol, media packaging
  moq-native/    - UniFFI-generated TypeScript + JSI TurboModule bindings
  moq-media/     - Platform native modules: camera capture, hardware codecs (Kotlin/Swift)
  moq-ui/        - Shared React Native components and Zustand state stores

Layer Responsibilities

Layer Technology What It Does
moq-core Rust, Quinn, tokio, UniFFI QUIC transport, MOQ protocol (draft-16/14), CMAF/MI packaging, NAL parsing, jitter buffer, A/V sync
moq-native uniffi-bindgen-react-native Auto-generated TypeScript + JSI bridge from Rust to RN
moq-media Kotlin (MediaCodec), Swift (VideoToolbox) Camera/mic capture, hardware H.264/Opus/AAC encode/decode
moq-ui React Native, Zustand, TypeScript Connection panel, stats overlay, shared hooks and stores

Data Flow

Publisher: Camera/Mic -> Platform hardware encode -> Rust (MOQ packaging) -> QUIC -> Relay

Subscriber: Relay -> QUIC -> Rust (MOQ depackaging) -> Platform hardware decode -> Video/Audio output

Media data stays on the native side. JS handles control plane only.

Protocol Support

  • MOQ Transport draft-16 (primary) with draft-14 fallback
  • MoQ Media Interop (draft-cenzano-moq-media-interop-03)
  • CMAF/fMP4 packaging
  • Codecs: H.264 (video), Opus (audio primary), AAC-LC (audio fallback)
  • Adaptive quality with 720p30 default

Prerequisites

  • Node.js 18+
  • Yarn 4+
  • Rust toolchain (rustup) with Android targets
  • Android SDK 34+ and NDK 27+ (for Android builds)
  • Xcode 15+ (for iOS builds)
  • cargo-ndk (cargo install cargo-ndk)

Setup

# Clone and install
git clone <repo-url> moq-rn
cd moq-rn
corepack enable
yarn install

# Build Rust core (verify compilation)
cd packages/moq-core
cargo build
cargo test
cd ../..

Building Rust UniFFI Bindings

The Rust core must be cross-compiled for each target platform using uniffi-bindgen-react-native. This generates the native shared library, TypeScript bindings, and C++ JSI turbo module glue.

# Add Android cross-compilation targets
rustup target add x86_64-linux-android    # emulator
rustup target add aarch64-linux-android   # physical device

# Set environment (use absolute paths)
export ANDROID_HOME=$HOME/Android/Sdk
export ANDROID_NDK_HOME=$ANDROID_HOME/ndk/27.1.12297006

cd packages/moq-native

# Build for physical device (arm64)
yarn ubrn build android --config ubrn.config.yaml --and-generate -t arm64-v8a

# Or build for emulator (x86_64)
yarn ubrn build android --config ubrn.config.yaml --and-generate -t x86_64

Important: The ubrn build replaces the jniLibs directory each time, so you can only target one ABI per build. Match the ABI to your target:

Target ABI ndk.abiFilters in build.gradle
Physical device arm64-v8a abiFilters "arm64-v8a"
x86_64 emulator x86_64 abiFilters "x86_64"

If switching targets, update packages/moq-native/android/build.gradle ndk { abiFilters } to match.

The build produces:

  • android/src/main/jniLibs/<arch>/libmoq_core.so -- the Rust shared library
  • src/generated/ -- TypeScript bindings
  • cpp/generated/ -- C++ JNI bridge code

If clang-format fails (cosmetic only), the bindings are still generated. Format manually with /usr/bin/clang-format -i cpp/generated/*.cpp cpp/generated/*.hpp.

Building for iOS

# Add iOS cross-compilation targets
rustup target add aarch64-apple-ios        # physical device
rustup target add aarch64-apple-ios-sim    # simulator (Apple Silicon)
rustup target add x86_64-apple-ios         # simulator (Intel)

cd packages/moq-native

# Build for physical device only
yarn ubrn build ios --config ubrn.config.yaml --and-generate --targets aarch64-apple-ios

# Or build for simulator only (Apple Silicon)
yarn ubrn build ios --config ubrn.config.yaml --and-generate --targets aarch64-apple-ios-sim

# Or build for all targets (creates universal xcframework)
yarn ubrn build ios --config ubrn.config.yaml --and-generate

Important: Unlike Android, iOS builds create an xcframework that can contain multiple architectures. Match the target to your build environment:

Target Architecture Use Case
Physical device aarch64-apple-ios Running on iPhone/iPad
Simulator (Apple Silicon) aarch64-apple-ios-sim Running on M1/M2/M3 Mac simulator
Simulator (Intel) x86_64-apple-ios Running on Intel Mac simulator

The build produces:

  • MoqRnMoqNativeFramework.xcframework -- Universal Rust framework for iOS
  • src/generated/ -- TypeScript bindings
  • cpp/generated/ -- C++ JSI bridge code

After building the Rust library, install CocoaPods dependencies:

cd apps/publisher/ios
pod install
cd ../../..

For the subscriber app, replace publisher with subscriber in the path above.

If clang-format fails (cosmetic only), the bindings are still generated. Format manually with /usr/bin/clang-format -i cpp/generated/*.cpp cpp/generated/*.hpp.

Post-build fix required: The generated C++ code uses namespace as a parameter name, which is a C++ reserved keyword. After running ubrn build ios --and-generate, fix it with:

sed -i '' 's/RustBuffer namespace,/RustBuffer namespace_,/g; s/RustBuffer namespace)/RustBuffer namespace_)/g' cpp/generated/moq_core.cpp

This must be reapplied each time you regenerate bindings. This is a known uniffi-bindgen-react-native issue.

Development

# Run publisher app
yarn publisher:start        # Metro bundler
yarn publisher:android      # Android build
yarn publisher:ios          # iOS build

# Run subscriber app
yarn subscriber:start
yarn subscriber:android
yarn subscriber:ios

Testing

Rust Unit Tests

cd packages/moq-core && cargo test

Android Physical Device

Connect the device via USB with developer mode and USB debugging enabled. Verify it appears:

adb devices -l
adb shell getprop ro.product.cpu.abi   # should show arm64-v8a

Ensure the Rust library was built for arm64-v8a (see "Building Rust UniFFI Bindings" above) and build.gradle has abiFilters "arm64-v8a".

Each app needs two terminals -- one for Metro bundler and one for the build:

# Terminal 1: Start Metro bundler
yarn subscriber:start

# Terminal 2: Build and install on device
yarn subscriber:android

For the publisher app, replace subscriber with publisher in the commands above.

Android Emulator

Ensure the Android SDK tools are on your PATH:

export ANDROID_HOME=$HOME/Android/Sdk
export PATH=$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH

Add these to ~/.bashrc or ~/.profile to make them permanent.

The x86_64 emulator requires KVM hardware acceleration. Check with kvm-ok or ls /dev/kvm. If unavailable, use a physical device instead.

List available emulators and start one:

emulator -list-avds
emulator -avd <avd-name> &

Wait for the emulator to fully boot. Ensure the Rust library was built for x86_64 and build.gradle has abiFilters "x86_64", then run the app as above.

On the emulator the publisher falls back to a test pattern generator since no camera is available. Audio capture uses the emulator's virtual microphone.

iOS Physical Device

Connect the device via USB and ensure it's trusted by your Mac. Verify it appears in Xcode's device list or via:

xcrun xctrace list devices

Ensure the Rust library was built for iOS (see "Building for iOS" above) and pods are installed.

Each app needs two terminals -- one for Metro bundler and one for the build:

# Terminal 1: Start Metro bundler
yarn publisher:start

# Terminal 2: Build and install on device
yarn publisher:ios

For the subscriber app, replace publisher with subscriber in the commands above.

iOS Simulator

Ensure Xcode is installed and a simulator is available:

xcrun simctl list devices available

Build and run the Rust library for simulator targets (aarch64-apple-ios-sim), then run the app as above.

On the simulator the publisher falls back to a test pattern generator since no camera is available. Audio capture uses the simulator's virtual microphone.

Integration Testing

End-to-end testing requires a MOQ relay running locally or remotely:

  1. Start the relay
  2. Launch the publisher app, connect to the relay, and start publishing
  3. Launch the subscriber app, connect to the same relay, fetch tracks, and subscribe

Project Status

Core implementation complete:

  • Monorepo scaffolding (yarn workspaces, RN 0.84)
  • Rust core: wire format, protocol messages, QUIC transport, session management
  • Rust core: H.264 NAL parsing, MoQ Media Interop, CMAF/fMP4, catalog
  • Rust core: jitter buffer, A/V sync timing
  • Rust core: draft-14 fallback support
  • UniFFI API surface and binding package
  • UniFFI cross-compilation and binding generation (ubrn build android)
  • Platform modules: Android (MediaCodec) + iOS (VideoToolbox)
  • Shared UI: Zustand stores, ConnectionPanel, StatsOverlay
  • WebTransport transport layer (default, with raw QUIC fallback)
  • PUBLISH/SUBSCRIBE message encoding (draft-16)
  • Publisher session: track management, SubgroupHeader streams, frame sending
  • Subscriber session: subscribe, receive loop, frame dispatch with cancellation
  • Expanded UniFFI API: publish, subscribe, catalog functions
  • Android: test pattern generator + Camera2 capture with fallback
  • iOS: test pattern generator + AVCaptureSession capture with fallback
  • Audio capture: Android (AudioRecord + AAC) + iOS (AVAudioEngine + AAC)
  • Native video views: VideoPreview + VideoPlaybackView (Android/iOS)
  • React Native hooks: useMoqSession, usePublisher, useSubscriber
  • Publisher app screen wired with capture and publish controls
  • Subscriber app screen wired with track selection and playback
  • On-device testing and optimization

Reference Projects

About

MOQ in React Native

Resources

Readme
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages