Skip to content

RealSeaberry/AudioFree

Repository files navigation

AudioFree — Real-Time System Audio Streamer for Android

Stream your Android device's system audio to one or more other Android devices with low latency and lossless PCM quality.

Dashboard   Telemetry   Settings

AudioFree supports two network modes — switch between them in Settings:

  • Local LAN mode — both devices on the same Wi-Fi router or hotspot. No extra software needed; lowest possible latency.
  • Tailscale mode — uses Tailscale, a zero-config mesh VPN built on WireGuard, giving each device a stable 100.x.x.x address that works across any network (home Wi-Fi, mobile data, different countries). All traffic is WireGuard-encrypted and NAT-traversing with no port-forwarding required.

Features

  • Lossless PCM audio — raw 16-bit samples, no codec degradation

  • Low latency — typically 30–80 ms on a direct hotspot link

  • Network mode toggle — switch between Local LAN and Tailscale VPN in one tap; receiver Dashboard shows this device's IP automatically

  • Two playback backends

    • Java modeAudioTrack with speed-based adaptive buffering
    • C++ / AAudio modeOboe with a lock-free SPSC ring buffer and hardware-exclusive stream
  • XOR Forward Error Correction (FEC) — recovers any single lost packet per 3-packet group; no retransmission needed

  • Adaptive jitter buffer — 3σ-based sliding-window target depth that tunes itself to current network conditions; 50 ms floor absorbs sudden path-latency step-changes (e.g. Tailscale direct → DERP relay switch)

  • P2P mode — one sender → one receiver

  • Host mode — one sender → multiple receivers simultaneously

  • Four quality presets

    Preset Sample Rate Channels Approx. bitrate
    High 48 000 Hz Stereo ~1.5 Mbps
    Medium 44 100 Hz Stereo ~1.4 Mbps
    Low 24 000 Hz Mono ~0.4 Mbps
    Minimal 16 000 Hz Mono ~0.3 Mbps
  • Real-time telemetry — buffer health curve and round-trip latency graph

  • Material 3 dark-theme UI built with Jetpack Compose


How It Works

┌─────────────────────────────────┐        UDP (LAN / Tailscale)
│           SENDER                │ ──────────────────────────────►
│                                 │   audio packets + FEC packet
│  AudioRecord (MediaProjection)  │   [A0][A1][A2][FEC] per 15 ms
│         ↓                       │
│  Capture loop (5 ms chunks)     │
│         ↓                       │
│  XOR FEC encoder                │
│         ↓                       │
│  Async UDP dispatch             │
│  (per-client non-blocking queue)│
└─────────────────────────────────┘

┌─────────────────────────────────┐
│           RECEIVER              │
│                                 │
│  UDP socket (receive loop)      │
│         ↓                       │
│  FEC group assembler            │
│  (recovers 1 lost packet/group) │
│         ↓                       │
│  Adaptive jitter buffer         │
│  (targetDepth = 3σ of jitter,   │
│   50 ms floor for step-changes) │
│         ↓                       │
│  AudioTrack  OR  Oboe engine    │
│  (Java mode)    (C++ AAudio)    │
└─────────────────────────────────┘

Packet format (UDP payload)

Byte  0        : type  (0 = audio, 1 = FEC)
Bytes 1–8      : group ID (uint64, big-endian)
Byte  9        : index within group (0–2 for audio, 3 for FEC)
Bytes 10–11    : payload length (big-endian)
Bytes 12–19    : sender timestamp (ms, uint64, big-endian)
Bytes 20–end   : raw PCM-16 payload

FEC scheme

Every three consecutive 5 ms audio chunks form a group. A fourth FEC packet is the byte-wise XOR of all three. If any single packet is lost, it can be recovered:

FEC = A0 ⊕ A1 ⊕ A2
A1_recovered = FEC ⊕ A0 ⊕ A2   (if A1 is missing)

Requirements

Requirement Details
Android version 10 (API 29) or higher — both sender and receiver
Network (LAN mode) Both devices on the same Wi-Fi router or hotspot — no extra software
Network (Tailscale mode) Tailscale installed and running on both devices
Permissions RECORD_AUDIO, FOREGROUND_SERVICE, INTERNET
Screen capture Sender requires a one-time MediaProjection grant

Note: AudioFree captures system audio (media playback, games, etc.) via the Android AudioPlaybackCapture API, which requires API 29+. It does not capture microphone input.


Getting Started

Build from source

  1. Clone the repository:

    git clone https://github.com/RealSeaberry/AudioFree.git
    cd AudioFree
  2. Open in Android Studio

  3. Let Gradle sync and download dependencies (requires internet access for first build).

  4. Build → Run on a physical device (emulators do not support AudioPlaybackCapture).

Quick setup — Local LAN (no Tailscale needed)

Both devices must be on the same Wi-Fi router or hotspot.

  1. Receiver device — go to Settings, set Network Mode to Local LAN, then return to Dashboard and tap Start Receiving. The receiver card shows this device's LAN IP (e.g. 192.168.1.5).

  2. Sender device — go to Settings, set Network Mode to Local LAN, enter the receiver's LAN IP shown above, choose P2P or Host mode, select audio quality, then tap Start Sending.

  3. Play any audio on the sender device — it streams to the receiver(s) instantly.

Quick setup — Tailscale (cross-network)

Use this when the devices are on different networks (different Wi-Fi, one on mobile data, etc.).

  1. Install Tailscale on all devices and sign in with the same account. Each device gets a stable 100.x.x.x address shown in the Tailscale app.

  2. Receiver device — go to Settings, confirm Network Mode is Tailscale VPN, then tap Start Receiving. The receiver card shows this device's Tailscale IP (e.g. 100.64.0.2).

  3. Sender device — go to Settings, confirm Network Mode is Tailscale VPN, enter the receiver's Tailscale IP shown above, choose mode and quality, then tap Start Sending.

Best-latency setup: hotspot + Local LAN mode

For the lowest possible latency (~20 ms RTT), use one device as a Wi-Fi hotspot and connect the other device to it, then use Local LAN mode. The two devices communicate directly over the hotspot link with no router, internet hop, or VPN overhead involved.

Tailscale can also be used over a hotspot link and still achieves ~20 ms because Tailscale establishes a direct peer-to-peer WireGuard tunnel between the two devices without leaving the local network.


Project Structure

app/src/main/
├── cpp/
│   ├── CMakeLists.txt          # NDK build config, links Oboe 1.9.0
│   └── native-lib.cpp          # C++ Oboe engine: SPSC ring buffer,
│                               #   onAudioReady callback, fade logic
└── java/com/example/audiofree/
    ├── MainActivity.kt         # Jetpack Compose UI (Dashboard / Telemetry / Settings)
    ├── AudioService.kt         # Foreground service lifecycle
    ├── AudioSender.kt          # Capture loop, FEC encoder, async UDP dispatch
    ├── AudioReceiver.kt        # UDP receive, FEC assembly, jitter buffer, playout
    ├── AudioMetrics.kt         # Shared StateFlow metrics (latency, buffer health)
    └── OboePlayer.kt           # JNI bridge to native Oboe engine

Key Technical Details

Sender (AudioSender.kt)

  • Zero-allocation capture loop — a pre-allocated ByteArray pool is read into directly by AudioRecord; no per-chunk heap allocation.
  • Non-blocking multi-client dispatch — in Host mode each extra receiver has its own bounded send queue drained by a dedicated coroutine, so a Wi-Fi stall on one client never blocks the capture pipeline.
  • Adaptive queue cap — capture → network queue sized at 150 ms to absorb transient Wi-Fi TX stalls without dropping groups.

Receiver (AudioReceiver.kt)

  • Pre-allocated payload pool (128 slots) — eliminates ~200 ByteArray allocations/sec in the hot receive loop that previously triggered GC stop-the-world pauses.
  • Online jitter measurement — inter-group first-arrival intervals tracked in a LongArray(30) ring buffer; variance computed incrementally (Var = E[X²] − (E[X])²), zero allocation per update.
  • Adaptive jitter buffer — 50 ms floor absorbs path-latency step-changes (e.g. Tailscale direct→DERP switch); spike detection immediately grows the target when a large delay interval is observed.
  • isHopeless threshold — proportional to targetJitterSize / fecGroupSize; adapts automatically to network conditions before declaring a group unrecoverable.
  • Silence injection — unrecoverable missing slots within a group receive PCM silence to avoid hard sample-value discontinuities (pops) at the audio/gap boundary.

C++ Oboe engine (native-lib.cpp)

  • Lock-free SPSC ring buffer (65 536 int16 samples ≈ 680 ms at 48 kHz stereo).
  • Pre-roll watermark (2 × hardware burst) prevents immediate re-underrun after starvation while minimising silence gap on recovery.
  • Linear fade-out at underrun onset; linear fade-in after STARVED → PLAYING transition — eliminates audible clicks.
  • Excess latency corrected by withholding writes and letting the ring buffer drain naturally — no hard PCM skip, no audible pop.
  • playbackJob runs on a dedicated thread at THREAD_PRIORITY_URGENT_AUDIO (with safe fallback if permission is denied) to prevent IO-thread-pool scheduling jitter from starving the ring buffer.

Settings Reference

Setting Description
Network Mode Local LAN (same Wi-Fi/hotspot, no extra software) or Tailscale VPN (cross-network, WireGuard encrypted)
Target IP IP of the receiver — shown automatically on the receiver's Dashboard card
Port UDP port — default 50000; control channel uses port+1
Connection Mode P2P (one receiver) or Host (multiple receivers via PING registration)
Audio Quality High / Medium / Low / Minimal — must match on both devices
Buffer Multiplier Multiplies AudioRecord internal buffer size; increase if capture drops occur on the sender
AAudio Mode Enables the C++ Oboe backend on the receiver for lower hardware latency

Dependencies

Library Version Purpose
Oboe 1.9.0 Low-latency C++ audio output
Jetpack Compose BOM 2023.08.00 UI framework
AndroidX Navigation Compose 2.7.6 In-app navigation
AndroidX Lifecycle KTX 2.7.0 Coroutine lifecycle integration

Limitations & Known Issues

  • Encryption: AudioFree sends raw UDP with no application-layer encryption. In Tailscale mode all traffic is protected by WireGuard end-to-end. In Local LAN mode, treat it as you would any unencrypted local stream — suitable for private home/office networks.
  • Audio capture: AudioPlaybackCapture only captures audio from apps that allow it (most media/game apps do; some DRM-protected content may be silent by policy).
  • Tailscale latency: expect 10–120 ms depending on whether Tailscale uses a direct peer-to-peer path or a DERP relay. Using a hotspot keeps it at ~20 ms; direct paths on the same network are comparable to plain LAN.
  • Host mode has been tested with up to 2 simultaneous receivers.

Contributing

Pull requests are welcome. For major changes please open an issue first to discuss what you'd like to change.


License

MIT

About

Stream your Android device's system audio to one or more other Android devices with low latency and lossless PCM quality.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors