A2DP audio glitches due to zero packets in output (IDFGH-1086)

[steenou]

Environment

• Module or chip used: ESP32-WROVER
• IDF version: v3.2-dev
• Build System: Make
• Compiler version: 1.22.0-80-g6c4433a
• Operating System: Linux
• Power Supply: external 3.3V

Problem Description

Using A2DP sink code examples results in glitches in the output. When putting a logic analyser on the I2S outputs of my ESP32-WROVER, I can see the glitches occur due to zeroes in the I2S data line. There is a pattern of zero packets on Left and Right data, always followed by 5 data packets (always Left Right Left Right Left), followed by zeroes again. These zeroes and data packets reoccur a couple times:

Sometimes there is a period of only zeroes after

The amount of clock cycles of the LRCLK (WS) during zeroes and during data is always the same as i2s_config.dma_buf_len which is declared in I2S.cpp. For example if I do i2s_config.dma_buf_len = 8, I see 5½ samples of zeroes, 2½ samples of data:

This is true for any value of i2s_config.dma_buf_len. n- 2½ samples of zeroes, 2½ samples of data, reoccurring.

The moments these glitches occur seem to occur when a Bluetooth A2DP stream has just started and randomly throughout the stream too. The glitches seem to occur both when the ESP32 is the I2S master and when the ESP32 is the I2S slave, although it seems to occur less during the stream when the ESP32 is the I2S master.

When the stream is paused, and resumed quickly after (maybe within a window of 3 seconds), there will be no glitches in the output. When the stream is paused, and you wait until
I (80520) BT_AV: A2DP audio state: Suspended has been written in the log, and then resume the stream (e.g. by pressing play again), you will hear glitches in the output.

Expected Behavior

Smooth audio when running ESP example code.

Actual Behavior

Glitchy audio.

Code to reproduce this issue

The a2dp_gatts_coex example code in the IDF.

Debug Logs

E (63693) BT_APPL: bta_av_rc_create ACP handle exist for shdl:0
I (63693) BT_AV: A2DP connection state: Connecting, [94:65:2d:20:56:8b]
I (63793) BT_AV: A2DP audio stream configuration, codec type 0
I (63803) I2S: PLL_D2: Req RATE: 44100, real rate: 44642.000, BITS: 32, CLKM: 14, BCK: 4, MCLK: 11289966.924, SCLK: 2857088.000000, diva: 64, divb: 11
I (63803) BT_AV: Configure audio player 0x21-0x15-0x2-0x35
I (63813) BT_AV: Audio player configured, sample rate=44100
W (63863) BT_APPL: new conn_srvc id:19, app_id:0
I (63863) BT_AV: A2DP connection state: Connected, [94:65:2d:20:56:8b]
I (63973) BT_AV: AVRC conn_state evt: state 1, [94:65:2d:20:56:8b]
I (63973) BT_AV: AVRC remote features 0x4b
I (63993) BT_AV: AVRC metadata rsp: attribute id 0x1, Red Eyes
I (63993) BT_AV: AVRC metadata rsp: attribute id 0x2, The War On Drugs
I (64003) BT_AV: AVRC metadata rsp: attribute id 0x4, Lost In The Dream
I (64003) BT_AV: AVRC metadata rsp: attribute id 0x20, 
I (70923) BT_LOG: bta_av_link_role_ok hndl:x41 role:1 conn_audio:x1 bits:1 features:x824b

W (70933) BT_APPL: new conn_srvc id:19, app_id:1
I (70933) BT_AV: A2DP audio state: Started
I (72553) BT_AV: Audio packet count 100
I (74023) BT_AV: Audio packet count 200
I (75473) BT_AV: Audio packet count 300
I (76933) BT_AV: Audio packet count 400
I (78383) BT_AV: Audio packet count 500
I (79833) BT_AV: Audio packet count 600
I (81283) BT_AV: Audio packet count 700
I (82733) BT_AV: Audio packet count 800
I (84183) BT_AV: Audio packet count 900
I (85243) BT_AV: A2DP audio state: Suspended
I (87853) BT_LOG: bta_av_link_role_ok hndl:x41 role:1 conn_audio:x1 bits:1 features:x824b

I (87853) BT_AV: A2DP audio state: Started
I (89483) BT_AV: Audio packet count 100
I (90943) BT_AV: Audio packet count 200
I (92393) BT_AV: Audio packet count 300
I (93853) BT_AV: Audio packet count 400
I (95303) BT_AV: Audio packet count 500
I (96763) BT_AV: Audio packet count 600
I (98213) BT_AV: Audio packet count 700
I (99663) BT_AV: Audio packet count 800

[redchenjs]

Hi @steenou, I found this line in your log that indicates you are using the PLL_D2 clock and the i2s real rate is 44642Hz which is higher than the expected 44100Hz:

I (63803) I2S: PLL_D2: Req RATE: 44100, real rate: 44642.000, BITS: 32, CLKM: 14, BCK: 4, MCLK: 11289966.924, SCLK: 2857088.000000, diva: 64, divb: 11

As far as I know, when i2s output clock is faster than the a2dp audio output speed, there will be some glitches after sometime, because the i2s tx buffer is underflow and the new data is not ready. My issue #3380 is just the opposite, I am using the APLL clock and the i2s real rate is lower than the expected one:

I (16536) I2S: APLL: Req RATE: 44100, real rate: 43945.238, BITS: 16, CLKM: 1, BCK_M: 8, MCLK: 11249981.000, SCLK: 1406247.625000, diva: 1, divb: 0

That causes BT_APPL: Pkt dropped problem after sometime because the i2s tx buffer is overflow and there is no place for new data to be stored.
Both of these issues are related to the inaccuracy of i2s clock. As metioned in #3380, after I enabled the workaround for the APLL clock, the BT_APPL: Pkt dropped problem disappeared, so maybe things will be different if you enable the APLL clock and apply the workaround.

[crespum]

Hi @redchenjs, the workaround you proposed eliminates indeed the packet drops that happen in the middle of the streaming. However, the zeroes described by @steenou appear only during the first few seconds after hitting play. From that moment on, sound works as expected. Is this happening to you or do you hear no glitches at any time?

In case it's helpful, we are using 32 bit samples (the APLL has been configured accordingly), dma_buf_count = 128 and dma_buf_len = 64.

[redchenjs]

@crespum I logged my i2s output by a logic analyzer, and yeah this problem did happen to me everytime at the first few seconds. So the inaccuracy of i2s clock only causes problems during the streaming, not at the beginning.

[steenou]

Good to know you have the same issue as me and @crespum. So all of us still have an issue that is tied to the DMA buffer when beginning a BT stream. @redchenjs I've been digging in the IDF a bit and I think I've found the code related to the I2S DMA buffer there. Do you know print the logs with I2S_TAG log level? We might see the glitching coincide with a log with the I2S tag...

[redchenjs]

@steenou We can set the Log output level in menuconfig to print the Debug level log, then clean & recompile the project:

But for ESP32, it seems too difficult to handle such a large amount of log output:

[crespum]

I've tested IDF 3.0, 3.0.2 and current master (df61612) and there are cut outs at the beginning of the streaming in all of them.

[pad52]

I confirm I have the same issue

[steenou]

The same glitch pattern occurs for me later on in the stream as well. It sounds like glitching, and can be triggered by increasing the distance between the phone and the ESP or when a different phone nearby goes in and out of airplane mode. There is no BT_APPL: Pkt dropped message in the monitor when this glitch happens so I'm not certain if it has to do with a straight-forward case of the phone being out of range or interference due to nearby devices

[steenou]

@pad52 @redchenjs do you have glitching sounds occurring during playback? Not only in the beginning?

[redchenjs]

@steenou yeah, it's very similar to what I've been through before, it's caused by the inaccurate i2s clock.

[steenou]

But I've applied your I2S patch and am still getting the glitches

[redchenjs]

@steenou Which I2S clock are you using now, PLL_D2 or APLL? This patch only works with APLL@44100Hz, 16bit, 2ch, chip Rev.1

[pad52]

@steenou Yes, I still have some glitches sometimes,
I confirm what @redchenjs wrote: If I tune the I2S Master clock to be very close to 44100 I manage to have almost no glitches.

1. manually tuning the I2S Master Clock - setting apll and with rtc_clk_apll_enable() as @redchenjs mentioned here (I2S APLL Clock is inaccurate at 44100Hz 16-bit 2ch (IDFGH-1058) #3380 (comment) while I find the correct values with the code that I posted in here : Unable to set I2S APLL Clock for 48khz 32bit 2ch (IDFGH-432) #2634 - take care to disable any call to i2s_set_clk() )
2. By setting wider dma buffers:
   .dma_buf_count = 12,
   .dma_buf_len = 1024,

With the spectrum anyway I can see some glitches sometimes, but not audibile.

[crespum]

Thanks @pad52. Our environment is a bit different, but we've modified the workaround accordingly. We use APLL@44100Hz, 16 data bits over 32 bits clock, 2ch, chip Rev.1.

if (sample_rate == 44100) {
    // see https://github.com/espressif/esp-idf/issues/3380
    rtc_clk_apll_enable(1, 28, 8, 5, 2);
    ESP_LOGW(
        BT_AV_TAG,
        "Workaround for I2S APLL clock at 44100Hz 32-bit 2ch");
}           

DMA buffers are:

.dma_buf_count = 128;
.dma_buf_len = 64;

[steenou]

@pad52 is that with a song playing or with a constant tone? I've found that you can really hear the glitching very well when you play a tone. I've made a recording of it, check it here

[pad52]

Well it happens with anything - of course with a tone it's easier to listen to it.
Yes it seems fom your recording the same problem that I had.

As @redchenjs wrote you need to tune the I2S Clock according to what are your needs..
and I will suggest to wide as much as possible the DMA buffers.
With this modifications I'm experiencing little glitches, not audible with sound.

My enviroment is:
ESP32 in Slave - Transmit mode. 2ch 16 bit expanded in 32 bit.
The Clock generator output a bit clock of 11289600 HZ
(23244100) * 4 Hz = 2822400 * 4 Hz = 11289600 Hz
(channelsbitfreq) * 4 = bit clock wanted.

tune this settings according to your clock generator (in case your esp is slave like mine)
Cheers

[crespum]

Just out of curiosity, where does your MCLK come from? According to the technical reference, the ESP32 does not support architectures where MCLK comes from an external XTAL (quite a constrain IMHO):

"When ESP32 I2S works in slave mode, the master must use I2Sn_CLK as the master clock [...]" (section 12.3).

[pad52]

Thanks @crespum that's really an interesting note!
I generate MCLK with another microcontroller and send to ESP32 only the Bit-Clock.
In fact - I have no audible glitches only if I set-up the clocks in ESP32 (using rtc_clk_apll_enable() ) to match the exact frequency of my generated clock.
So maybe it's still usable with external master clock altough not the best idea.

[steenou]

Hey @pad52, what values do you use in rtc_clk_apll_enable()?

[fknrdcls]

APLL:
Just use the fixed i2s_apll_calculate_fi2s from here: #2634 (comment)

Slave MCLK:
From memory the I2S module only needs enough MCLK to make the serial logic tick, the MCLK is set internally there is no MCLK input, incorrect MCLK in slave mode does not cause drift unless the I2S engine doesn't have enough clock cycles to work at the input sample rate. Best to set it correctly though.

More relevantly then:
Transmitting audio/video which is sampled synchronously via an async transmission will always suffer from drift, the send and receive devices are not going to be 100% the same clock no matter how good your PLL is so drift will always happen. You would need to implement one of the solutions that I believe are in the spec. Basically you speed up or slow down your playback clock based on your buffer timings to avoid under-run / overrun. This should be part of the stack though? Never looked into it as that codec is rubbish anyway.

I'm willing to bet though if you ran two ESP32's BT tx/rx in slave mode from the same I2S BCLK source or one master and one slave (no MCLK connection just bit clock LR clock and your data) you'd have perfect sync - and if not you are dropping BT packets.

[crespum]

Thanks @fknrdcls. Have you also experienced glitches the first few seconds after the playback starts?

[fknrdcls]

@crespum from memory when I tried the A2DP sink code example it behaved exactly as you described. I remember tweaking the sample-rate manually to attempt to match the data input rate (we're talking 10s of Hz) and it made it better not perfect, as I said above I decided the problem was just the mismatch in the audio sampling rate on the source device and playback rate. If the ESP32 had AAC or aptx I likely would have stuck it out and fixed it, but it's just not worth it for A2DP. Also as I alluded to above, the bluetooth stack must all be complete and therefore should handle the playback as expected, there might be something missing or broken in the sample code or the drivers.

I'm not sure how we're supposed to take the ADF seriously if (and I haven't checked recently so I may be mistaken) if the I2S driver still can't set the APLL with correct values. Frankly the ESP32 shouldn't be considered for bluetooth audio, A2DP is just not good enough, and AAC decode should be possible and I might be willing to bother porting if we're given proper ISA documentation. Decode would be fine in 32bit float with the tight loops rolled out into optimised assembly to take advantage of the multiply-add multiply-subtract and fine tune the float coprocessor pipelining.

[jason-mao]

Hi guys

Firstly, fixed inaccurate i2s MCLK modification will be available soon.
For @steenou issue, setting wider enough DMA buffer is makes sense, and recommend that provide a ring buffer between bt_a2d_sink_data_cb and i2s_write , refer to: https://github.com/espressif/esp-adf/blob/7d6f6ce69318335657733b0bd1566e9bdacd9bf3/components/audio_service/bluetooth_service.c#L174-L181

[eric-gitta-moore]

Hello, how do you implement the connection to the Bluetooth headset? I use ESP32 to connect to the Bluetooth headset and always stuck in "esp_a2d_source_connect". I just modified the official example a2dp_source. But connecting a computer can be, that is, connecting a Bluetooth headset does not work. Sometimes it suddenly crashes and then restarts and suddenly connects.