Performance issues (audio stuttering) during 24-bit FLAC and MP3 decoding on ESP32-S3 (N16R8)

[yty7690-bot]

Hi,

I am currently developing an audio decoding system using the arduino-audio-tools library on an ESP32-S3 (N16R8 module, 16MB Flash / 8MB PSRAM). While I have successfully configured the multi-decoder and connected the FLAC decoder, I am experiencing severe audio stuttering issues and would appreciate some advice.

[Current Status & Symptoms]

Hardware: I am using the ESP32-S3 N16R8 model, so memory (PSRAM) capacity should be more than sufficient.

Ultimate Goal: To achieve smooth and stable playback of 24-bit FLAC audio files.

Symptoms: * FLAC Playback: The audio does not play properly and stutters severely

MP3 Playback: It plays relatively smoother than FLAC, but intermittent stuttering or glitches still occur during playback.

[Questions & Concerns]

Is it achievable to smoothly play 24-bit FLAC files given the hardware capabilities of the ESP32-S3 chipset itself? Is this a problem that can be resolved by optimizing library buffer configurations or CPU core allocations?

If this is a hardware or performance limitation within the arduino-audio-tools environment, should I consider pivoting the project toward ESP-ADF (Espressif Audio Development Framework)? I would love to hear from anyone who has experienced significant performance improvements with 24-bit FLAC decoding by switching to ESP-ADF.

""
#include "Arduino.h"
#include "WiFi.h"
#include "dlna.h"
#include "AudioTools.h"
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/Communication/AudioHttp.h"
#include "AudioTools/Disk/AudioSourceURL.h"
#include "AudioTools/AudioCodecs/CodecHelix.h"
#include "AudioTools/Concurrency/RTOS.h"
#include "AudioTools/Concurrency/AudioPlayerThreadSafe.h"
#include "AudioTools/AudioLibs/I2SCodecStream.h"
#include "AudioTools/AudioCodecs/CodecFLACFoxen.h"
#include "ogg.h"

const char* ssid = "help!";
const char* password = "help!";

// DLNA
const int port = 9000;
WiFiServer wifi(port);
HttpServer<WiFiClient, WiFiServer> server(wifi);
UDPService udp;
DLNAMediaRenderer media_renderer(server, udp);

// AudioPlayer
URLStream url;
AudioSourceDynamicURL source(url);

I2SStream out;

MultiDecoder multi_decoder;
AACDecoderHelix dec_aac;
MP3DecoderHelix dec_mp3;
WAVDecoder dec_wav;
FLACDecoderFoxen dec_flac;
AudioPlayer player(source, out, multi_decoder);

// Multitasking
QueueRTOS queue(20, portMAX_DELAY, 5);
AudioPlayerThreadSafe player_save(player, queue);
Task dlna_task("dlna", 8000, 10, 0);

void onEOF(AudioPlayer& player) {
if (source.size() > 0) {
Serial.println("*** onEOF() ***");
player_save.end();
source.clear();
media_renderer.setPlaybackCompleted();
}
}

void handleMediaEvent(MediaEvent ev, DLNAMediaRenderer& mr) {
switch (ev) {
case MediaEvent::SET_URI:
Serial.print("Event: SET_URI ");
Serial.println(mr.getCurrentUri());
source.clear();
source.setTimeoutAutoNext(1000);
player_save.setPath(mr.getCurrentUri());
player_save.begin(0, media_renderer.isActive());
break;
case MediaEvent::PLAY:
Serial.println("Event: PLAY");
player_save.setActive(true);
break;
case MediaEvent::STOP:
Serial.println("Event: STOP");
player_save.end();
url.end();
break;
case MediaEvent::PAUSE:
Serial.println("Event: PAUSE");
player_save.setActive(false);
break;
case MediaEvent::SET_VOLUME:
Serial.print("Event: SET_VOLUME ");
Serial.println(mr.getVolume());
player_save.setVolume(static_cast(mr.getVolume()) / 900.0);
break;
case MediaEvent::SET_MUTE:
Serial.print("Event: SET_MUTE ");
Serial.println(mr.isMuted() ? 1 : 0);
player_save.setMuted(mr.isMuted());
break;
default:
Serial.println("Event: OTHER");
}
}

void setupWifi() {
WiFi.begin(ssid, password);
WiFi.setSleep(false);
Serial.print("Connecting to WiFi ..");
while (WiFi.status() != WL_CONNECTED) {
Serial.print('.');
delay(1000);
}
WiFi.setSleep(false);
Serial.println("connected!");
}

void setup() {
// setup logger
player_save.setVolume(0.05);
Serial.begin(115200);
DlnaLogger.begin(Serial, DlnaLogLevel::Warning);
AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Warning);

// start Wifi
setupWifi();
//url.setReadBufferSize(256 * 1024);
Serial.println("starting I2S...");
auto config = out.defaultConfig(TX_MODE);
config.sample_rate = 44100;
config.channels = 2; //
config.bits_per_sample = 16; //
config.pin_bck = 14;
config.pin_data = 13;
config.pin_ws = 12;
// config.buffer_count = 32;
//config.buffer_size = 2048;

out.begin(config);

// setup MultiDecoder
multi_decoder.addDecoder(dec_aac, "audio/aac");
multi_decoder.addDecoder(dec_aac, "audio/mp4");
multi_decoder.addDecoder(dec_mp3, "audio/mpeg");
multi_decoder.addDecoder(dec_wav, "audio/wav");
multi_decoder.addDecoder(dec_flac, "audio/flac");

// configure player: EOF handling
player.setOnEOFCallback(onEOF);
player.setAutoNext(false);

// setup media renderer (use event callbacks to handle audio at app level)
media_renderer.setBaseURL(WiFi.localIP(), port);
Serial.print(WiFi.localIP());
Serial.println(port);
media_renderer.setMediaEventHandler(handleMediaEvent);

// start device
if (!media_renderer.begin()) {
Serial.println("MediaRenderer failed to start");
}

dlna_task.begin( {
media_renderer.loop();
});
}

void loop() {
// if we have nothing to copy, be nice to other tasks
if (player_save.copy()==0) delay(200);
}
""

[pschatzmann]

You are doing way too much, so it is difficult to say if the issue has been introduced by your code!

Your goal should be to identify the playback bottleneck first!

My gut feeling is that your esp32 network stack is too slow to provide the required amout of data...

• If you have the chance to test with a SD card, try to test with SDMMC: If this is working, the issue is definitly the network.
• I would recommend to start with the simplest decoding sketch as possible.
• Try it with a http url first: https adds a lot of additional overhead
• Try to replace the URLStream with URLStreamBuffered (and play around with the buffer size): Buffering the encoded data is much more efficient that having a big i2s buffer which consists of decoded data.
• If you stick to http, you can also try the URLStreamESP which avoids the Arduino network stack and uses the one provided by IDF
• Try with PSRAM switched off: RAM is much faster than PSRAM, so use it only if it is necessary
• Try to test the network speed e.g. with a MeasuringStream class and be realistic about your expectations:

Format	Typical Bitrate
MP3 (128 kbps)	128 kbps
MP3 (192 kbps)	192 kbps
MP3 (320 kbps)	320 kbps
FLAC (CD quality, 16-bit/44.1 kHz)	700–1,100 kbps
FLAC (24-bit/96 kHz)	2,000–4,500 kbps