Custom Codecs

By default, the A2DP Sink and the A2DP Source use the SBC codec that is built into ESP-IDF. Starting with ESP-IDF 5.5, you can replace it with your own codecs from the audio-tools library:

• The A2DP Sink decodes the received audio with the decoders that you register via add_decoder().
• The A2DP Source encodes your PCM data with the encoders that you register via add_encoder().

Each decoder or encoder that you add registers its own stream endpoint (SEP). The codec that is actually used is negotiated with the connected device. This lets you support AAC in addition to SBC.

If you add no decoder or encoder, nothing changes: the library uses the built-in SBC codec, as before.

Requirements

• ESP-IDF >= 5.5 and the audio-tools library
• The codec libraries you want to use, e.g.:
  • arduino-libsbc for SBC (SBCDecoder, SBCEncoder)
  • arduino-libhelix for AAC decoding (AACDecoderHelix)
  • arduino-fdk-aac for AAC encoding and decoding (AACEncoderFDK, AACDecoderFDK)
• The following sdkconfig (menuconfig) settings under "Component config > Bluetooth > Bluedroid Options":

CONFIG_BT_A2DP_USE_EXTERNAL_CODEC=y
CONFIG_BT_A2DP_CODEC_AAC_ENABLED=y   # only needed for AAC
CONFIG_BT_A2DP_SEP_NUM_MAX=2         # >= number of add_decoder()/add_encoder() calls

AAC stream endpoints are only supported by ESP-IDF >= 6.1. On older versions the AAC registration fails, which is reported in the ESP_A2D_SEP_REG_STATE_EVT log, and SBC is negotiated instead.

The Arduino IDE gives you no sdkconfig to edit, so you need to build with ESP-IDF (with arduino-esp32 as a component) or with PlatformIO and a sdkconfig.defaults.

Note: when CONFIG_BT_A2DP_USE_EXTERNAL_CODEC is active, ESP-IDF no longer includes its internal SBC codec. So you must register at least one decoder (Sink) or encoder (Source).

A2DP Sink: Custom Decoders

A2DPDecoderSBC and A2DPDecoderAAC wrap any audio_tools::AudioDecoder that you give them, so you can choose the implementation. Include the codec header yourself.

The decoded PCM goes through the regular output pipeline of the sink, so I2S/AudioTools output, volume control, the stream reader callbacks and write_audio() all keep working.

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecSBC.h"
#include "AudioTools/AudioCodecs/CodecAACHelix.h"
#include "BluetoothA2DPSink.h"
#include "A2DPDecoderSBC.h"
#include "A2DPDecoderAAC.h"

I2SStream i2s;
BluetoothA2DPSink a2dp_sink(i2s);

SBCDecoder sbc_decoder;
A2DPDecoderSBC a2dp_sbc(sbc_decoder);
AACDecoderHelix aac_decoder;
A2DPDecoderAAC a2dp_aac(aac_decoder);

void setup() {
  a2dp_sink.add_decoder(a2dp_aac);
  a2dp_sink.add_decoder(a2dp_sbc);
  a2dp_sink.start("MyMusic");
}

void loop() { delay(1000); }

A2DPDecoderAAC advertises AAC-LC only. Call set_he_aac(true) before start() if your decoder can also handle HE-AAC and HE-AAC v2.

You can call mime() to find out which codec is active, e.g. "audio/sbc" or "audio/aac".

The full sketch is in the bt_music_receiver_codec example.

A2DP Source: Custom Encoders

A2DPEncoderSBC wraps an audio_tools::SBCEncoder. It advertises the SBC capabilities and configures the encoder with the parameters negotiated with the sink: blocks, subbands, allocation method and bitpool. The optional second constructor argument limits the bitpool (default 53).

A2DPEncoderAAC wraps any audio_tools::AudioEncoder that produces AAC-LC (e.g. AACEncoderFDK). The second constructor argument is the bitrate that is advertised to the sink, which should match the bitrate you configured in the encoder. If the encoder produces ADTS, the ADTS headers are removed, because the Bluetooth stack adds the LATM framing itself.

The sink can negotiate a lower bitrate than the one you advertise. To apply the negotiated bitrate to your encoder, register a callback with set_bitrate_callback(): it is called before the encoder is started.

You provide the audio data as before, via set_data_callback(), set_data_callback_in_frames() or an Arduino Stream. The PCM data stays 16 bit stereo, so the existing callbacks and the volume control keep working:

• The sample rate is defined with set_sample_rate() on the encoder (default 44100), and only this rate is advertised to the sink.
• If the sink negotiates mono, the data is downmixed automatically.

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecSBC.h"
#include "AudioTools/AudioCodecs/CodecAACFDK.h"
#include "BluetoothA2DPSource.h"
#include "A2DPEncoderSBC.h"
#include "A2DPEncoderAAC.h"

BluetoothA2DPSource a2dp_source;

SBCEncoder sbc_encoder;
A2DPEncoderSBC a2dp_sbc(sbc_encoder);
AACEncoderFDK aac_encoder;
A2DPEncoderAAC a2dp_aac(aac_encoder, 192000);

// apply the bitrate negotiated with the sink
void set_aac_bitrate(uint32_t bitrate) {
  aac_encoder.setBitrate(bitrate);
}

int32_t get_data_frames(Frame *frame, int32_t frame_count) {
  // fill the frames with your stereo audio data
  return frame_count;
}

void setup() {
  aac_encoder.setBitrate(192000);
  a2dp_aac.set_bitrate_callback(set_aac_bitrate);
  a2dp_source.add_encoder(a2dp_aac);
  a2dp_source.add_encoder(a2dp_sbc);
  a2dp_source.set_data_callback_in_frames(get_data_frames);
  a2dp_source.start("MySpeaker");
}

void loop() { delay(1000); }

add_encoder() must be called before start(). It returns false if CONFIG_BT_A2DP_USE_EXTERNAL_CODEC is not enabled; in that case the source keeps using the built-in SBC encoder.

After the codec has been negotiated, mime() returns the active codec and get_audio_info() returns the negotiated sample rate and channels.

How the data is sent

ESP-IDF does not request data from the application when an external codec is used. Instead, a dedicated encode task:

1. reads the PCM data from your callback,
2. encodes it,
3. packs the frames into media packets that fit into the MTU of the connection, and
4. sends them with esp_a2d_source_audio_data_send().

The task paces itself in real time. If your callback provides no data, silence is sent so that the sink does not disconnect.

You can tune it with the following defines (see config.h):

Define	Default	Description
A2DP_MANAGED_ENCODE_TASK_STACK	8192	Stack size of the encode task: AAC encoding needs much more (e.g. 40000)
A2DP_MANAGED_ENCODE_TASK_PRIO	tskIDLE_PRIORITY + 5	Priority of the encode task
A2DP_MANAGED_ENCODE_CHUNK_SIZE	512	PCM bytes requested from the data callback per loop
A2DP_MANAGED_ENCODE_LEAD_MS	100	How far (ms) the encoding may run ahead of real time
A2DP_MANAGED_ENCODE_MAX_LAG_MS	200	If the encoding falls further behind than this (ms), the pacing is reset
A2DP_MANAGED_ENCODE_SEND_RETRY_MS	100	How long (ms) to retry when the transmit queue is full before a packet is dropped

The full sketch is in the bt_music_sender_codec example.

Adding Support for Another Codec

Subclass A2DPDecoder (see A2DPDecoder.h) or A2DPEncoder (see A2DPEncoder.h) and implement:

• codec_type() and mime()
• build_capability(): the capabilities that are advertised for the stream endpoint
• parse_audio_info(): determines the sample rate and channels from the negotiated configuration
• for encoders, also samples_per_frame(): used for the packet timestamps
• for encoders, optionally max_frames_per_packet(), payload_header_size() (the bytes of the MTU used by the media payload header that the stack adds) and configure(): applies the negotiated parameters to the encoder

The Bluetooth stack has to support the codec type as a stream endpoint. Currently ESP-IDF supports SBC and AAC.