Enjoying this project? It is built and tested in my free time. A small
donation helps buy test hardware and keeps the development going. Thank you! ❤️
Wireless 5.1 / 7.1 surround sound from an Apple TV, built from low-cost ESP32-S3 boards: one Gateway, and one board per speaker.
The Apple TV sees one AirPlay 2 speaker: the Gateway. The Gateway takes the complete surround stream and passes it on, unchanged, to up to eight speaker boards over your Wi-Fi. Every speaker decodes only its own channel (Front Left, Centre, Subwoofer, Rear Right, …) and plays it through any amplifier and speaker you like. All speakers share one clock taken from the Wi-Fi access point itself, so they play each sample at the same moment. No cables between the speakers, no AV receiver, one web page for the whole system.
- 🎬 5.1 and 7.1 AAC (Apple Music Dolby Atmos, films and series from the Apple TV apps).
- 🎵 Stereo too: ALAC 44.1 kHz/16-bit and lossless 48 kHz/24-bit, AAC 44.1 / 48 kHz, from any AirPlay 2 sender. A stereo song plays on the front pair and the subwoofer, as on an AV receiver.
- 📡 One AirPlay target. You pick the Gateway, not eight speakers. Pairing, RTSP and the Apple TV's clock are handled once, by the Gateway.
- ⏱️ Microsecond clock alignment between speakers through the access point's beacon timer (TSF). No extra hardware, no cables.
- 🎤 Room calibration with an optional INMP441 microphone on the Gateway: distance, delay and level of every speaker measured from your seat, applied with one click.
- 🎛️ One page for all speakers: role, level and delay of every speaker, and a live view of what each one plays. Each speaker keeps its own 2 × 24-band parametric EQ / crossover, loudness and latency settings.
- 🔀 Format changes on the fly: the Apple TV can switch between stereo and surround from track to track; the speakers follow without reconnecting.
- ⚡ About a third of one CPU core per speaker for 7.1, thanks to a patched Fraunhofer FDK AAC decoder that synthesises just one channel.
- 🔗 Speakers belong to the system: a speaker is not an AirPlay
receiver of its own and never shows up on an iPhone; it just follows the
Gateway. (A board that is its own AirPlay receiver is the
standalonebranch.)
Note
The project is under active development. A Gateway and two speakers play a 7.1 stream from an Apple TV, each its own channel, audibly in sync. Features, defaults and wiring may still change, and this README is updated with every change. See Limitations.
- Gallery
- How it works
- What you need
- Assembly and wiring
- Installation
- First start
- Settings
- Technical details
- Measuring sync
- Limitations
- Troubleshooting and logs
- Licenses and credits
Click an image to open it full size.
The Apple TV sends surround sound over AirPlay 2 only to a receiver that says it accepts it. The Gateway says so, receives the whole multichannel stream and shares it with the speakers:
- The Gateway is the only AirPlay receiver. It announces 5.1 and 7.1, so the Apple TV sends it the full multichannel stream. mDNS, HAP pairing, RTSP and the Apple TV's PTP clock all run there.
- It forwards, it does not decode. Every instruction from the Apple TV (format, key, start, flush, pause, volume, timing) is passed to the speakers over a small control protocol, and every encrypted audio packet is copied byte for byte to each speaker. The slowest speaker paces the stream through TCP, so the Gateway needs no audio buffer of its own.
- Every speaker decodes only its own channel. One ESP32-S3 cannot decode all eight channels in real time, and it does not need to. A speaker is not an AirPlay receiver: it finds the Gateway by mDNS, decrypts the stream with the key the Gateway passed on and plays its channel on both outputs of its DAC or amplifier (L = R), so you can drive one speaker, two drivers (woofer + tweeter) or a subwoofer amplifier.
- One clock for all. Every board on the same Wi-Fi access point keeps a copy of the access point's beacon timer (TSF). Eight times a second the Gateway tells the speakers "the Apple TV's time P is TSF T". Each speaker converts T to its own clock and plays every sample at the moment the Apple TV scheduled it.
You choose each speaker's role (Front Left, Centre, Subwoofer, …) on the Gateway's page, even while music is playing. Start with two or three speakers and add the rest later.
┌─────────────────────────────┐
Apple TV ──AirPlay──► Gateway (ESP32-S3) │
(PTP clock) │ pairing · RTSP · PTP │
│ no decoder, no audio output │
└─────┬──────────────┬─────────┘
control + time │ │ the same audio packets,
(TCP, port 7100) │ unchanged, to every speaker
┌──────────┬────────┴─┬────────────┴┬────────────┐
▼ ▼ ▼ ▼ ▼
ESP32 ESP32 ESP32 ESP32 ESP32 (… up to 8)
Front L Front R Centre Subwoofer Rear L/R
│ │ │ │ │
amp amp amp amp amp
│ │ │ │ │
speaker speaker speaker subwoofer speaker
| What | Measured |
|---|---|
| TSF reading jitter per board | ~4.5 µs rms, averaged by a 64-point line fit |
| Clock difference speaker ↔ speaker | a few µs (from the fits; scope test pending) |
| Playout error of each speaker | within ±0.4 ms, corrected slowly by the I2S servo |
| Speaker ↔ speaker in practice | 0.1–0.3 ms, about 3–10 cm of sound travel |
| Transport delay of the time messages | 3–60 ms, does not matter by design |
The time messages carry a TSF timestamp rather than "now", so how long they take over Wi-Fi does not enter the result.
| Source | Front L / R | Centre | Subwoofer | Side / Rear | Mono mix |
|---|---|---|---|---|---|
| 7.1 (Apple Music Dolby Atmos) | own channel | own channel | LFE channel | own channel | all except LFE |
| 5.1 (films, series) | own channel | own channel | LFE channel | Side L/R; Rear plays Side | all except LFE |
| Stereo (music, YouTube) | left / right | silent | mix of L and R | silent | mix of L and R |
With stereo the subwoofer gets the full-range mix; a low-pass on its EQ page (or the subwoofer amplifier's own crossover) keeps the bass.
| Part | Notes |
|---|---|
| One ESP32-S3 board, the same type as the speakers (N8R8) | It has no audio output: no DAC, no amplifier, no wiring. A USB supply is all it needs. |
| A place with a good signal | It receives the stream from the Apple TV and sends it again to every speaker, so it carries the most Wi-Fi traffic. Near the access point is best. |
| Optional: INMP441 I2S microphone | For room calibration: VDD → 3V3, GND and L/R → GND, SCK → GPIO 5, WS → GPIO 6, SD → GPIO 4 (changeable in menuconfig). |
| Part | Notes |
|---|---|
| ESP32-S3 board with 8 MB flash and 8 MB PSRAM (N8R8), e.g. ESP32-S3-DevKitC-1 N8R8 | PSRAM is required: the stream buffer alone takes 6 MB. Boards without PSRAM (or with 2 MB) will not work. |
| Audio output, one of: | |
| • an I2S amplifier such as MAX98357A | Simplest: digital in, speaker out, about 3 W. Good for small satellites and surrounds. |
| • an I2S DAC such as PCM5102A + any analogue amplifier | Any size: mini (PAM8403, 2 × 3 W), medium or powerful (TPA3110 / TPA3116, TDA7498, a plate amplifier for a subwoofer…), one- or two-channel. |
| Speaker(s), 4–8 Ω | One full-range driver, or a woofer + tweeter pair (see Option B). |
| Power supply | 5 V USB (≥ 1 A) for the ESP32-S3; a separate supply for a larger amplifier (e.g. 12 V ≥ 2 A, as your amplifier requires). |
| Enclosure | 3D-printed, wooden or an existing speaker box. |
| Jumper wires or a small PCB | 5 wires for the I2S connection. |
| System | Boards | Positions |
|---|---|---|
| 5.1 | 1 Gateway + 6 speakers | Front L, Front R, Centre, Subwoofer, Side L, Side R |
| 7.1 | 1 Gateway + 8 speakers | the 5.1 positions + Rear L, Rear R |
| Start small | 1 Gateway + 2–3 speakers | e.g. Front L + Front R (+ Centre) |
A channel without a speaker is simply not played. The centre carries most of the dialogue, so build it early.
- An Apple TV for 5.1 / 7.1 (surround is sent only by the Apple TV; an iPhone, iPad or Mac sends stereo, which the speakers play as well).
- A 2.4 GHz Wi-Fi network with good coverage at every speaker (the ESP32-S3 has no 5 GHz radio).
- All boards on the same access point. The shared clock is that access point's beacon timer: each access point, and each node of a mesh system, has its own. A speaker on another access point logs a warning and stays silent.
The Gateway needs no wiring (only the optional calibration microphone, see What you need). Everything below is for the speakers.
Note
The GPIOs below are the defaults and can be changed in menuconfig
(see Installation). Wiring options may be extended in
future versions; this section will be updated.
| ESP32-S3 | MAX98357A |
|---|---|
| GPIO 5 | BCLK |
| GPIO 6 | LRC (WS) |
| GPIO 4 | DIN |
| 5V | VIN |
| GND | GND |
- Speaker to SPK+ / SPK− (4–8 Ω).
- Leave SD and GAIN unconnected (default: (L+R)/2 output, 9 dB gain). Because both I2S slots carry the same channel, the result is that channel at full level.
- At full volume the amplifier draws up to ~1 A from 5 V: use a good USB supply and short power wires.
| ESP32-S3 | PCM5102A |
|---|---|
| GPIO 5 | BCK |
| GPIO 6 | LCK (LRCK / WS) |
| GPIO 4 | DIN |
| 5V | VIN |
| GND | GND |
| — | SCK: not used (connect to GND, or close the module's SCK bridge) |
On the common purple PCM5102A module check the solder jumpers on the back: FLT = L, DEMP = L, XSMT = H (otherwise the DAC stays muted), FMT = L (I2S).
The DAC's L OUT and R OUT carry the same speaker channel, which lets you use any amplifier, small or powerful, one- or two-channel:
| Amplifier | Connection | Setup on the speaker's EQ page |
|---|---|---|
| One channel (mono amp, a single side of a stereo amp, a subwoofer plate amp) | L OUT → amp input | nothing; for a subwoofer add a low-pass filter if the amp has none |
| Two channels, one full-range speaker | L OUT → amp, use one channel | nothing |
| Two channels, active 2-way (woofer + tweeter) | L OUT → CH1 → woofer, R OUT → CH2 → tweeter | a crossover: low-pass on the Left output, high-pass on the Right output |
| Powered speaker / active subwoofer | L/R OUT → its line input | nothing |
Active 2-way crossover (one example, adjust to your drivers):
- Open
http://<speaker-ip>/eqand tick Enable EQ and preamp. - Left Output Filters (woofer): two LP filters at the crossover frequency (e.g. 2500 Hz), Q 0.707 each. Two stacked Butterworth filters make a 24 dB/oct Linkwitz-Riley slope.
- Right Output Filters (tweeter): two HP filters at the same frequency, Q 0.707 each.
- Tweeters are usually louder: lower the Right preamp until both drivers sound balanced. Add peak or shelf filters to correct each driver's response.
- Apply & Save: the change is live and stored, no restart needed.
Warning
Always set the tweeter's high-pass before playing loud. Full-range bass destroys a tweeter in seconds. The per-output Mute buttons on the EQ page help while setting up.
Power: the ESP32-S3 and the DAC run from USB 5 V; a larger amplifier gets its own supply (e.g. 12 V ≥ 2 A). Connect the grounds of the DAC and the amplifier (the signal ground), and keep the amplifier's power wires away from the I2S wires.
| Function | Default GPIO | Notes |
|---|---|---|
| Amplifier enable / standby | 21 | switches the amplifier on while audio plays; active high or low |
| WS2812 RGB VU LED | 38 | the on-board LED of many DevKitC boards |
| Wired latency measurement (ADC loopback) | 7 | DAC output → coupling capacitor + bias divider → ADC pin |
| Sync test pulse (also on the Gateway) | 9 | for an oscilloscope, see Measuring sync |
The firmware is built with ESP-IDF v6.0.2. Follow Espressif's
installation guide,
then open a terminal with the ESP-IDF environment loaded
(source ~/esp-idf/export.sh, or the path where you installed it).
The Gateway is the main branch of the surround repository (the
earlier design, where every board is its own AirPlay receiver, is the
standalone branch):
git clone https://github.com/Kristian8606/esp32-airplay2-surround.git
cd esp32-airplay2-surround| Folder | Builds | Firmware file |
|---|---|---|
| the repository folder | the Gateway | build/airplay-gateway_<version>.bin |
speaker/ |
the speakers | speaker/build/airplay-speaker_<version>.bin |
| Code | Where |
|---|---|
| the Gateway's own (AirPlay receiver: mDNS, pairing, RTSP; proxy, relays, speaker registry, speakers page) | main/ |
| the speaker's own (audio engine, link to the Gateway, EQ page) and FDK AAC | speaker/main/, speaker/components/ |
| shared (Wi-Fi, web server, settings, OTA, logs, clock, sync protocol) | common/, data/www/ |
| menuconfig options of both | components/airplay_options/ |
Both use the shared settings in sdkconfig.defaults; the Gateway adds
sdkconfig.defaults.gateway. Each is an ordinary ESP-IDF project: go into
its folder and run idf.py as usual.
Run idf.py menuconfig in the folder of the firmware you configure (the
repository folder for the Gateway, speaker/ for the speakers). Both show
the same menus; some options matter for one firmware only:
| Menu | What you can set | For |
|---|---|---|
| Surround Build Role | the role (set by the folder: leave it); a fixed Gateway address for networks that filter mDNS; the sync test pulse | both |
| AirPlay Receiver | setup Wi-Fi name and password; playback buttons and AirPlay model (the icon senders show) | Wi-Fi both, the rest Gateway |
| Multichannel Experiment | 5.1 / 7.1 on or off, default speaker position | both |
| I2S Playout | BCLK / LRCK / DATA / MCLK GPIOs, output latency of your DAC, wired latency test; web admin password | speakers (password both) |
| Amplifier Power Control | amplifier enable GPIO and level | speakers |
| RGB Audio/VU LED | LED on/off and GPIO | both |
| AirPlay Logs and Diagnostics | log detail | both |
Every speaker runs the same firmware: its role is set later on the Gateway's page.
Connect a board over USB.
Speakers (repeat for every speaker):
cd speaker
idf.py -p /dev/cu.usbmodem101 build flash monitorGateway, from the repository folder:
idf.py -p /dev/cu.usbmodem101 build flash monitorOn macOS list the ports with ls /dev/cu.*; on Windows the port is COMx,
on Linux usually /dev/ttyACM0. Every board's log goes to its USB port
(USB Serial/JTAG, so idf.py monitor shows it) and to its /logs page.
Later versions are installed over Wi-Fi:
- Gateway: on its main page, Firmware Update → choose
build/airplay-gateway_<version>.bin→ Upload Firmware. - Speakers, all at once: on the Gateway's speakers page, panel
Speaker firmware → Choose .bin →
speaker/build/airplay-speaker_<version>.bin→ Upload. The Gateway keeps it in its otherwise unusedstoragepartition (1.9 MB) and every speaker with another build installs it by itself: one at a time and right away, also a speaker that was switched off and comes back later. A speaker that is playing stops its stream for the update (the others play on), checks size and SHA-256 and restarts into the new firmware at once; in a buffered stream (music, films) it plays again from the next track, skip or seek, in a realtime stream at once. Nothing waits for the playback to end; only a running room calibration holds the update back. Each tile showswill update,updating…orupdate failed(after three tries). If a speaker restarts and does not come back with the new build within 3 minutes, the update stops there (update stopped) instead of going on to the next speaker; check that speaker, then upload the firmware again to continue. A new speaker firmware that cannot reach the Gateway (crashes at boot, no link) goes back to the previous one at its next restart (app rollback). The rollback needs the bootloader of v4.5.11 or later: flash every speaker once over USB (idf.py flashinspeaker/). - One speaker by hand: its own main page, Firmware Update, as for the Gateway.
The web interface is part of the firmware and is updated with it.
Important
Speakers older than v4.5.9 cannot update from the Gateway and need this
once by hand on their own page. From v4.5.9 on the Gateway also updates
speakers whose control protocol differs from its own (protocol version …, Gateway speaks … in its log), so a new Gateway brings the speakers up to it.
A speaker image larger than the storage partition is refused on upload.
- Connect every board to Wi-Fi. A new board opens the access point
ESP32-AirPlay-Setup. Join it with a phone or laptop; the setup page opens by itself (or go tohttp://192.168.4.1/). Choose your network, enter the password, Connect to WiFi. - Name the boards. Find each IP address in your router (or the serial
log), open
http://<board-ip>/and set its name (AirPlay Device Name on the Gateway, Speaker Name on a speaker). The Gateway's name is what the Apple TV shows; clear speaker names (Front Left, Centre, …) make the speakers page easy to read. - Speakers connect by themselves. A speaker finds the Gateway on the network and shows up on the Gateway's speakers page; there is nothing to switch on.
- Give every speaker its role. Open the Gateway's page
http://<gateway-ip>/speakers(or Open Speakers on its main page), pick a role, level and delay for each speaker, Save. - Choose the Gateway on the Apple TV. Open Control Center, select the AirPlay / audio output icon and tick only the Gateway. The speakers are not listed there; they follow the Gateway.
- Play something in surround (Apple Music Dolby Atmos or a film). The speakers page shows what every speaker plays right now, e.g. AAC · 48 kHz · 24-bit · 7.1 · plays: Front Left.
Tip
A speaker that joins or restarts during a film stays silent until the next skip or seek (the Apple TV then restarts the stream at a point every speaker can start from). During screen mirroring it joins at once.
| Position | Where to put it | Notes |
|---|---|---|
| Front Left / Right (L / R) | ~30° left / right of the screen | main pair; music and effects |
| Centre (C) | above or below the screen | most of the dialogue |
| Subwoofer (LFE) | anywhere low, often near a corner | the low-frequency effects channel; with stereo, the L+R mix |
| Side Left / Right (Ls / Rs) | beside the listener, ~90–110° | the surround pair of 5.1 |
| Rear Left / Right (Lrs / Rrs) | behind the listener, ~135–150° | 7.1 only; with a 5.1 stream they play Side L / R |
| Mono mix | anywhere | every channel except LFE in one speaker; uses the most CPU |
Important
With 5.1 / 7.1 the Subwoofer plays only the LFE channel (the extra bass effects of films), not the bass of the other speakers. There is no bass management, so the front speakers should reproduce reasonable bass on their own. With music the LFE channel is often silent; with films it is busy.
Everything is configured in the browser.
| Card | What it does |
|---|---|
| Surround Speakers | opens the speakers page below |
| Playback | play/pause, next, previous and volume of the current AirPlay sender |
| WiFi Network | scan and change the network |
| Device Settings | the AirPlay name the Apple TV shows |
| Firmware Update | OTA update with a .bin file |
| System | IP, MAC, memory, firmware, uptime, reset reason, Wi-Fi signal and channel; restart |
| System Logs / WiFi Speedtest | links to the pages below |
The Gateway has no audio output, so it has no EQ or latency cards.
At the top, a room map seen from above: the screen at the top, you in
the middle and all eight positions around you. An empty position is dashed,
a position with a speaker is outlined, and a speaker that is playing glows.
Two speakers on the same position are marked orange. Click a position to
jump to its speaker. Above the map: what is playing now
(AAC · 48 kHz · Stereo) and how many positions are taken.
Below it, one tile per connected speaker, side by side:
| Field | What it does |
|---|---|
| Role | which channel it plays: Front Left … Rear Right, or Mono mix |
| Level (dB) | trim from −30 to 0 dB; turn the louder speakers down, the quietest stays at 0 dB |
| Delay (ms) | 0–50 ms; about 2.9 ms per metre a speaker is closer to the listener than the farthest one |
| Live status | coloured tags: format, rate and bits (AAC · 48 kHz · 24-bit), layout (7.1, 5.1, Stereo), the channel actually played, buffered or realtime stream, playout sync and I2S correction, buffer fill, TSF quality |
| Links | the speaker's own page and its EQ |
Every speaker keeps its settings in its own flash and applies a change at once, also while playing. A warning shows when two speakers have the same role.
| Card | What it does |
|---|---|
| Surround Speaker | a reminder that role, level and delay are set on the Gateway's page |
| WiFi Network / Device Settings | network, and the speaker's name shown on the Gateway's page |
| Output Latency | delay of your hardware after the ESP (DAC, DSP), so all speakers stay in sync; Measure runs a wired loopback test |
| Firmware Update / System | OTA update, board status, restart |
| Audio EQ / System Logs / WiFi Speedtest | links to the pages below |
| Card | What it does |
|---|---|
| Surround speaker (5.1 / 7.1) | role of this speaker (the same setting as on the Gateway's page; a change here shows there too) |
| Loudness | ISO 226 loudness compensation: restores bass and treble at low volume |
| Left / Right Output Filters | two independent chains of up to 24 biquad filters each: peak, low-pass, high-pass, band-pass, notch, low shelf, high shelf; a preamp and a mute button per output |
| Apply & Save | applies live and stores in flash |
An optional INMP441 microphone on the Gateway (wiring in The Gateway) measures every speaker from your seat and suggests its delay and level, in about 2 s per speaker:
- Put the Gateway with its microphone at your listening seat, at ear height (a power bank will do). Keep the room quiet.
- Test mic checks the microphone and shows the room noise; −60 dBFS or quieter is good. Music may keep playing meanwhile.
- Calibrate: if the Apple TV is connected, it is disconnected for the measurement (the speakers can only chirp between streams); afterwards pick the Gateway again as its audio output. Each speaker in turn plays four short chirps (300 Hz–8 kHz, 20 ms; the subwoofer 30–150 Hz, 200 ms) through its own EQ, so a 2-way speaker's crossover protects the tweeter: at −30 dBFS first, louder (up to −12 dBFS) only if the microphone does not hear them clearly.
- The table shows every speaker's distance, suggested delay (the farthest gets 0 ms, the others 2.9 ms per metre they are closer) and level (the quietest gets 0 dB, the others are turned down to it). Apply to speakers saves them like a manual edit; Discard changes nothing.
| In the table | Meaning |
|---|---|
| ± cm after a distance | how much the four chirps disagree; a few cm is normal |
| ± dB after a level | the same for the level |
| Level not repeatable (… dB over the chirps): measure again | the four levels differ by more than 2 dB, usually from room noise; measure again in quiet |
| Not heard clearly | the chirps did not stand out of the noise even at −12 dBFS, or their timing disagrees by more than 0.5 ms |
| Hint: the chirp came back inverted | + and − may be swapped (or a 2-way speaker's tweeter is wired so); only a hint, nothing is changed |
| Subwoofer: level and delay by ear | its low chirp gives a delay, but no level |
Two runs in a quiet room should agree within a few centimetres and about 0.5 dB; if they do not, measure again before applying.
How it measures:
- Travel time without cables. A speaker plays its chirps when it can and reports when each one left it, on the access point's TSF (its output latency included); the Gateway timestamps its recording on the same TSF. The difference is the sound's travel time.
- The direct sound, not an echo. The arrival is the first correlation peak within 6 dB of the strongest one: the direct sound always comes first, while a reflection off a wall can be louder.
- Level as the ear hears it: the RMS of the chirp and the next 10 ms of the room (full-range speakers above 200 Hz), the median of the four.
No signal from the microphone: check its wiring means the Gateway clocks the microphone but reads a constant value: check VDD → 3V3, GND, L/R → GND (otherwise it sends on the other channel), SD → GPIO 4, SCK → GPIO 5, WS → GPIO 6. Without a microphone signal the speakers still play their chirps one after another (so you hear that each one works), and the panel says nothing was measured.
/logs: live log of the board over WebSocket./speedtest: ping, download and upload between the browser and the board, to check Wi-Fi at the speaker's (or the Gateway's) position.
Security: with a Web UI admin password set in menuconfig, firmware
updates, restart, Wi-Fi and name changes and the latency test require the
user admin and that password.
The Gateway advertises these formats and the Apple TV chooses one per stream; the speakers decode whatever the Gateway passes on:
| Transport | Codec | Rate | Channels | Frames / packet |
|---|---|---|---|---|
| Realtime UDP (type 96) | ALAC 16-bit | 44.1 kHz | 2 | 352 |
| Buffered TCP (type 103) | ALAC 16-bit | 44.1 kHz | 2 | 352 |
| Buffered TCP (type 103) | ALAC 24-bit (lossless) | 48 kHz | 2 | 352 |
| Buffered TCP (type 103) | AAC-LC | 44.1 / 48 kHz | 2 | 1024 |
| Buffered TCP (type 103) | AAC-LC | 48 kHz | 6 (5.1) / 8 (7.1) | 1024 |
- The Apple TV decides the format from the formats the Gateway announces and, for each track, from what the content offers. An "Atmos" label in Apple Music does not guarantee a 7.1 stream; some tracks arrive as stereo. YouTube sends stereo only.
- Format changes mid-stream. Every audio packet carries its format in the RTP SSRC. A speaker switches codec, channels, bit depth or rate per packet, without a new SETUP. A codec or channel change at the same rate keeps the queued audio; a rate change (44.1 ↔ 48 kHz) restarts I2S at the right moment with a short gap, and the timeline stays continuous. AirPlay labels AAC "F24"; that is the stream profile, not the source's depth.
5.1 and 7.1 arrive as AAC-LC with channel configuration 6 or 7. A speaker reads the channel labels of every stream and finds its role by name, so one setting works for 5.1 and 7.1 and survives format changes.
| Stream | Channel order (AAC elements) |
|---|---|
| 5.1 (config 6) | C · L R · Ls Rs · LFE |
| 7.1 (config 7) | C · L R · Ls Rs (side) · Lrs Rrs (rear) · LFE |
With 5.1 the Rear positions play Side L / R, so a 7.1 set of speakers also works with 5.1 content.
Note
7.1 naming. The names above follow Apple's 7.1 order (first pair front, second pair side, back pair rear). The ISO reading of configuration 7 names the first pairs differently. A listening test with a 7.1 channel test signal is in progress; if Front and Side turn out swapped, a setting will be added and this README updated.
With a stereo stream every speaker decodes both channels and takes what its role needs: Front L / R play left / right, Subwoofer and Mono mix play (L + R) / 2, Centre, Side and Rear stay silent. 5.1 and 7.1 streams are never touched by this.
- AAC is decoded with Fraunhofer FDK AAC, patched to synthesise only the selected channel: the bits of all channels are read, but only the element that holds the selected speaker is inverse quantised, filtered and transformed. Output is signed 32-bit, straight to float, without a 16-bit step. ALAC uses Espressif's decoder; 24-bit samples also go straight to float.
- DSP in 32-bit float: stereo routing, EQ / crossover, loudness, volume (the Apple TV's volume, then the speaker's level trim).
- I2S with 32-bit samples at the stream's rate: 44.1 or 48 kHz for stereo, 48 kHz for surround. The selected speaker goes to both slots (L = R).
The Apple TV schedules every packet on its PTP clock: each packet carries an RTP timestamp and an instruction says which PTP time belongs to which timestamp. Only the Gateway takes part in PTP. The speakers get the Apple TV's time through the access point:
- Every Wi-Fi station keeps a copy of its access point's TSF, a microsecond timer carried in every beacon. Each board reads its TSF together with its own clock and fits a line through the last 64 such readings (outliers dropped), which averages out the reading jitter.
- Eight times a second the Gateway sends TIME: a straight line on the TSF, "the Apple TV's time is P at TSF T and runs at 1 + rate", with the clock's identity and state.
- A speaker looks up its own local time at that TSF and gets the offset to the Apple TV's clock, and carries it forward on the rate until the next message. Because the message names a TSF moment rather than "now", its delay over Wi-Fi does not matter.
One smooth line for all speakers. While the Gateway relays audio most of
the Apple TV's PTP Syncs reach it 5 to 100 ms late, and even the best of a
few seconds scatter by 3 to 6 ms. A Sync is never early, so every sample lies
on or below the true sender time. The speakers get a line fitted to the raw
Syncs (main/gw_timeline.c): the best Sync of each second over the last ten
minutes, and the edge of their upper envelope (convex hull) over the middle
of that window, the lowest line above all of them. Its rate comes from timely
Syncs minutes apart (a few ppm); late Syncs never count, and nothing guesses
clock steps from them. The published line follows it on that rate, takes a
difference back at most 100 µs per second and bends at most 5 ppm per
second. A Sync more than 1 s below the fit, or the first one after 5 s
without any (the Apple TV restarted), starts the fit again at once. The line
steps only for a new sender clock, a restarted access point or a jump beyond
20 ms; after a step it follows the fit directly for 10 s. Every speaker
evaluates the same line, so all of them play the same time even when one
receives an update later than another. The Gateway logs the line every 10 s:
TIME … | line epoch N rate … ppm (fitted), envelope … ms from it over … s, PTP filter … ms from it.
4. From there each speaker runs the playback engine of an AirPlay 2
receiver: it plays every sample at the scheduled time, minus its
output latency, plus its delay from the speakers page, and tunes
its I2S clock live to follow the Apple TV. The line's rate on the
speaker's own clock is exactly the I2S correction needed, so the I2S
clock runs at it directly and only the remaining phase is steered (back
over 10 s): a new track, a reboot or a bend of the line needs no
relearning. The speaker logs SERVO LINE: line … ppm, sync … ms -> I2S … ppm every 10 s.
Wi-Fi dropouts. During a stream a speaker carries the last line forward
for up to 30 s, and both sides wait 20 s (instead of 6 s) before they give a
silent link up. A dropout of a few seconds costs nothing: the speaker plays on
from its buffer, in step, and logs Gateway silent … and TIME again after … ms. Only a longer one ends that speaker's stream; it joins again at the
next pause, seek or track.
| Stream | How the Gateway passes it on |
|---|---|
| Buffered (type 103: music, films) | one TCP connection per speaker, a copy of the Apple TV's. TCP backpressure makes the slowest speaker pace the Apple TV, so nobody falls behind and the Gateway keeps no buffer. Speakers keep up to ~90 s of compressed audio. The fill is paced at twice real time after a 2 s start: sent at full speed to every speaker at once, it kept the Wi-Fi so busy that the Apple TV's clock packets arrived late. |
| Realtime (type 96: screen mirroring, system audio) | every UDP audio and sync packet is copied to each speaker's ports. A speaker that misses a packet asks the Gateway; several requests for the same packet reach the Apple TV once. |
A speaker that connects in the middle of a session gets the session's settings replayed (format, key, volume, timing). With a buffered stream it waits for the next flush (skip or seek), because it can only start decoding at a point every speaker starts from; with a realtime stream it joins at once.
| Gateway | Speaker (one channel of 7.1) | |
|---|---|---|
| CPU | relays only, no decoding (not yet measured) | ~31–34 % of one core (measured on a rear speaker) |
| Audio buffer | none (TCP backpressure) | 6 MB compressed stream buffer in PSRAM |
| Decode task stack | — | 20 KiB in PSRAM (deepest use ~11 KiB) |
| Free PSRAM during 7.1 | — | ~400 KiB |
| Firmware image | ~1.26 MB | ~1.79 MB (no AirPlay receiver code) |
| Board | Core 0 | Core 1 |
|---|---|---|
| Gateway | RTSP, PTP | Wi-Fi, LwIP, buffered and realtime relays |
| Speaker | AAC decoding | Wi-Fi, LwIP, stream reader, I2S playout, status / LED / log tasks |
The FDK changes (single-channel synthesis, 32-bit output, hardware bit
counting on the ESP32-S3, element skipping, smaller buffers, lazy limiter)
are listed in
speaker/components/fdk_aac/MODIFICATIONS.md.
| Part | Where | What it does |
|---|---|---|
| AirPlay receiver | main/rtsp, main/hap, main/plist, main/mdns_airplay.c |
mDNS, pairing, RTSP; Gateway only (a speaker is not an AirPlay receiver) |
| Shared services | common/network |
Wi-Fi, web server core, OTA, logs, the PTP clock (full on the Gateway, fed by the Gateway on a speaker) |
| Web pages | main/web_gateway.c, speaker/main/web_speaker.c |
each firmware's own pages and APIs on top of the shared web server |
| Gateway proxy | main/gw_proxy.c |
implements the audio engine API on the Gateway and forwards each call (FWD … in the log) |
| Speaker registry | main/gw_speakers.c |
control link on port 7100 (_apsurround._tcp), late joins, settings, status, TIME messages, which speaker updates next |
| Speaker firmware | main/gw_fw.c, speaker/main/spk_update.c |
the image in the Gateway's storage partition (gw_fw); download, check and install on the speaker (spk_update) |
| Buffered relay | main/gw_relay.c |
copies the TCP audio stream to every speaker, paced by the slowest (RELAY …) |
| Realtime relay | main/gw_rt_relay.c |
copies UDP audio and sync packets, passes retransmit requests on (RT RELAY …) |
| Time | common/surround/tsf_fit.c, main/gw_timeline.c, speaker/main/spk_time.c |
TSF line fits on both sides; the Gateway's smooth sender-time line; feeds the speaker's clock (TIME …, gw_timeline) |
| Speaker link | speaker/main/spk_link.c |
finds the Gateway, runs every forwarded call locally (CMD …) |
| Protocol | common/surround/surround_proto.h |
message types and payloads, versioned |
| Audio engine | speaker/main/audio |
FIFO, FDK AAC / ALAC decode, PCM store, I2S playout and servo |
| Room calibration | main/gw_calib.c, main/gw_mic.c, common/surround/cal_chirp.c |
microphone capture, chirp search, suggestions (gw_calib in the log); the speaker side plays the chirps (ROOM CAL …) |
-
Enable Surround Build Role → Sync test pulse on the Gateway and on two speakers (GPIO 9 by default). Each board raises the pin for 10 ms at every whole second of the Apple TV's clock, as it sees it, and logs
PULSE … interrupt late avg … max …. -
Put the three pins on an oscilloscope, trigger on the Gateway, play anything. Gateway → speaker is a speaker's clock error; speaker → speaker is the difference the listener would hear before playback adds its own.
-
For the full chain give both speakers the role Mono mix, play a click track and compare the two DAC outputs:
ffmpeg -f lavfi -i "aevalsrc='if(lt(mod(t,1),0.001),0.8*sin(2*PI*4000*t),0)':s=48000" -t 600 -ac 2 -c:a aac -b:a 256k clicks.m4a
- Experimental. Tested with one Apple TV, a Gateway and two speakers playing together (three connected): 7.1 and stereo AAC, skip, seek, pause, volume, role and level changes while playing, a second sender taking over.
- Room calibration is tested with an INMP441 and up to six speakers. An echo more than 6 dB louder than the direct sound (a speaker behind an obstacle) still misleads its timing, and the subwoofer's level is left to the ear.
- Not yet tested: realtime (screen mirroring, system audio) through the Gateway, the stereo routing to front pair and subwoofer, oscilloscope measurements, more than three speakers, a weak Wi-Fi signal.
- Late joins: a speaker that connects during a film waits for the next skip or seek; until then it stays silent.
- One access point. All boards must be associated with the same access point (on a mesh, the same node).
- Playout servo: each speaker holds its own playout within ±0.4 ms; two speakers differ by 0.1–0.3 ms. That is inaudible as an echo but could be tightened for a perfectly fixed stereo centre.
- No bass management: with 5.1 / 7.1 the Subwoofer plays only LFE.
- The 7.1 Front / Side naming is not yet confirmed (see Channels).
- Mono mix decodes all channels and may not keep up with 7.1.
- A switch between 44.1 and 48 kHz has a short silence while I2S is reconfigured.
- Rear and side channels are often nearly silent in music; they come alive with films and Atmos mixes.
- Some video apps take 20–30 s from the start of AirPlay to the first audio (seen with HBO); that is the sender.
| Problem | Check |
|---|---|
A speaker does not appear on /speakers |
Same Wi-Fi as the Gateway; its log says connected to Gateway …. If mDNS is filtered, set Surround Build Role → Gateway IPv4 address on the speaker. |
| A speaker still shows up as an AirPlay speaker on an iPhone | It runs an old firmware from before the Gateway design: update it. |
| Which build does a speaker run? | Its tile shows fw v4.5.5 7c5d1e74: the version plus a code that changes with every build. A speaker on a different build than the others is marked; the Gateway's own is under the page title. |
| A speaker connects and drops again | protocol version …, Gateway speaks … in the Gateway log: update both to the same firmware. |
| A speaker is silent after joining | Wait for the next skip or seek (speaker … joins at this FLUSH). Then its role on /speakers, the Mute buttons on its EQ page, the PCM5102A XSMT jumper, the BCLK / LRCK / DATA wiring. |
no common TSF, no sender time on a speaker |
It is on another access point (or mesh node) than the Gateway. |
| No sound at all after a Gateway update or restart | The Apple TV dropped the session: select the Gateway again as the audio output. |
| The Apple TV sends stereo only | The track or app may only offer stereo (YouTube, some "Atmos" tracks); the speakers page shows the layout. |
| Subwoofer silent with surround music | Normal: most music has no LFE. Try a film. |
| Dropouts | Wi-Fi signal at that speaker and at the Gateway (/speedtest, RSSI on the main page). |
| One speaker early or late | Its Delay on /speakers; its Output Latency (DSP, external processors). |
Every board streams its log to http://<board-ip>/logs. Useful lines:
| Line | Where | Meaning |
|---|---|---|
FWD set_format AAC 48000 Hz … |
Gateway | an Apple TV instruction forwarded to the speakers |
RELAY rx … | paced … ms | … RELAY tx Front:live … |
Gateway | audio in from the Apple TV and out to each speaker; how long the fill was held back |
RT RELAY data … pkts -> N speakers | NACK … |
Gateway | realtime audio and retransmit requests |
control connection from … / speaker … joins at this FLUSH |
Gateway | a speaker connected / started playing |
TIME tsf fit 64 samples -31.0 ppm rms 4.5 us … locked=1 |
both | TSF fit quality and the Apple TV clock's state |
ROLE: Surround speaker (follows the Gateway) |
speaker | boot banner of the speaker firmware |
connected to Gateway … |
speaker | the control link is up |
CMD start_buffered: ESP_OK, audio port … |
speaker | a forwarded instruction run locally |
sender time from the Gateway: gm=… |
speaker | the speaker has the Apple TV's clock |
audio_status: AAC | 48000 Hz | … sync=+0.06ms | i2s=-51ppm … |
speaker | playback, its sync error and I2S correction |
NOW PLAYING | AAC-LC | 48000 Hz | 24-bit (F24) | 7.1 (8ch) | Front Right (R) |
speaker | the channel actually played |
stereo sources: silent (role Centre (C)) |
speaker | what this role plays from a stereo song |
MC CPU … avg …% of one core |
speaker | decoding time per frame |
For deeper debugging enable AirPlay Logs and Diagnostics → Log every RTSP
request and response in menuconfig on the Gateway (verbose; turn it off
afterwards).
- Fraunhofer FDK AAC (
speaker/components/fdk_aac/fdk-aac) is under the Fraunhofer FDK license,NOTICE. The copy here is a Third-Party Modified Version of the Fraunhofer FDK AAC Codec Library for Android; the changes and their dates are listed inMODIFICATIONS.md. The license allows redistribution in source and binary form without copyright fees but grants no patent rights: AAC patent licenses are available through Via Licensing or the patent owners. - The rest of this repository has no license file yet, so no license is granted for it beyond viewing it on GitHub.
- Other components (ESP-IDF,
esp_audio_codec, libsodium, mDNS, cJSON, led_strip) come through the ESP-IDF component manager under their own licenses.
Grows out of the standalone branch of
esp32-airplay2-surround,
where every board is its own AirPlay receiver playing one channel, and
airplay-esp32, the stereo
AirPlay 2 receiver for the ESP32-S3. AAC decoding through
fdk-aac.
AirPlay, Apple TV and HomePod are trademarks of Apple Inc. This project is not affiliated with or endorsed by Apple.
If this project helped you build your surround system, please consider
supporting it. 🙏






