Skip to content

Communication AOO

Jump to bottom
pschatzmann edited this page Oct 25, 2026 · 4 revisions

AoO - Audio over OSC is a message based real-time audio streaming protocol that uses Open Sound Control as its wire format: it sends audio from arbitrary senders (sources) to arbitrary receivers (sinks) over UDP. Unlike plain UDP the receiver knows the format, buffers the data as jitter buffer, requests lost packets again, compensates the clock drift and can mix several senders. It is the protocol of the official aoo library, which is also available for Pure Data and SuperCollider.

The implementation is provided by the separate arduino-aoo library, which is based on the AudioTools. See Communication for the general considerations.

Installation

cd ~/Documents/Arduino/libraries
git clone https://github.com/pschatzmann/arduino-audio-tools.git
git clone https://github.com/pschatzmann/arduino-aoo.git

For Opus you also need the arduino-libopus library.

The classes

Class Description
AOOSender an audio sink (AudioOutput): encodes (PCM by default, or any AudioTools encoder, e.g. Opus) and sends the audio; answers resend requests, pings, invitations
AOOReceiver an audio source (AudioStream): receives, buffers and decodes the audio of one or several senders and mixes them; pull the audio with a StreamCopy
AOOReceiverTask same as AOOReceiver, but the messages are received in a separate task (ESP32: on the other core)
AOOStreamUDP UDP transport (unicast or multicast)
AOOStreamSerial serial transport with HDLC framing

Sending

#include "AudioTools.h"
#include "AOO.h"

AudioInfo info(48000, 1, 16);
SineWaveGenerator<int16_t> sineWave;
GeneratedSoundStream<int16_t> sound(sineWave);
Throttle throttled(sound);  // the generator is faster than real time
AOOStreamUDP udp(IPAddress(192, 168, 1, 44), 9999);  // receiver
AOOSender sender(1, udp);   // source id 1
StreamCopy copier(sender, throttled);

void setup() {
  // connect to WiFi ...
  sineWave.begin(info, N_B4);
  throttled.begin(info);
  auto cfg = sender.defaultConfig();
  cfg.copyFrom(info);
  // cfg.redundancy = 2;  // send each block twice
  sender.begin(cfg);
}

void loop() { copier.copy(); }

Receiving

#include "AudioTools.h"
#include "AOO.h"

AudioInfo info(48000, 2, 16);  // output format: all senders are converted to it
AOOStreamUDP udp(9999);
AOOReceiver receiver(1, udp);  // sink id 1
I2SStream i2s;
StreamCopy copier(i2s, receiver);

void setup() {
  // connect to WiFi ...
  auto cfg_i2s = i2s.defaultConfig(TX_MODE);
  cfg_i2s.copyFrom(info);
  i2s.begin(cfg_i2s);
  auto cfg = receiver.defaultConfig();
  cfg.copyFrom(info);
  receiver.begin(cfg);
}

void loop() { copier.copy(); }

To check the received data you can replace the I2SStream with a CsvOutput (paced with a Throttle): use a low sample rate on a microcontroller, because a CSV line per frame needs a lot of bandwidth.

Opus

Register the codec on both sides; like the official library one Opus stream per channel is used:

#include "AudioTools/AudioCodecs/CodecOpusMultiStream.h"

OpusMultiStreamAudioEncoder opus;  // sender
opus.config().default_channel_mapping = OPUS_CHANNEL_MAPPING_SEPARATE;
opus.config().frame_sizes_ms_x2 = OPUS_FRAMESIZE_20_MS;
sender.setEncoder("opus", opus);

receiver.addDecoder("opus", []() {  // receiver: a new decoder for each sender
  auto* decoder = new OpusMultiStreamAudioDecoder();
  decoder->config().default_channel_mapping = OPUS_CHANNEL_MAPPING_SEPARATE;
  return (AudioDecoder*)decoder;
});

On the ESP32 increase the loop stack size for Opus (e.g. SET_LOOP_TASK_STACK_SIZE(30000)) and use complexity = 1 on the sender.

Error correction

The receiver primes its buffer (buffer_depth blocks) to half before it starts the playback and fills missing blocks with silence. Lost blocks are requested again (resend requests, default), and the sender can send each block several times (redundancy): resend requests cost almost no bandwidth and work well with Opus and 20 ms frames, redundancy is the most robust against random loss but multiplies the bandwidth. The clock drift is compensated by adaptive resampling. See Choosing the Error Correction for the measurements and recommendations.

Blocks which do not fit into max_packet_size (default 960 bytes) are split into several UDP packets: keep it below the MTU of your network, because the ESP32 drops fragmented packets.

Interoperability

The implementation has been tested in both directions against the official aoo library (PCM and Opus, on the desktop and with an ESP32-S3 over WiFi): e.g. you can receive the audio of a microcontroller in Pure Data or SuperCollider, or send audio from there to a microcontroller. Use the same ids and ports on both sides (the examples use sink id 1 on port 9999 and the source port 9998, like the official examples).

Examples and documentation

Clone this wiki locally