Basic throughput and buffering question (ESP32, Signal generator into I2S)

[bjoerngiesler]

Hi all,

this is a very stupid very basic question so please don't shoot me.

I'm setting up a signal generator into an I2S output. (Not the final application but I'm starting small.) Audio format is 44100Hz, 1 channel, 16 bit. With that format, the output should be able to consume 88200 bytes per second, or 88.2 bytes per millisecond.

The I2S output has an input buffer size of 3072 bytes, the signal generator can of course produce much more. In my loop, I copy 3072 bytes to the I2S stream, then vTaskDelay(1). The copy to the I2S stream takes 1.7ms give or take, giving a throughput of 1800 bytes per millisecond. There is no throttling, the 3072 bytes stay constant. As far as I can see, the write() method in the I2S stream just hands the data off to ESP32's i2s_write() and returns the result.

If the audio driver can only consume 88 bytes per millisecond, but the stream throughput is 1800 bytes per millisecond, where do the remaining 1712 bytes go? Who buffers or drops them?

Happy to be enlightened... :-)

[pschatzmann]

The overall processing logic is quite simple: The sine generator is an asynchonous data source, so it provides the data as fast as possible. The max generation speed is thus mainly driven by the selected implementation.

The ESP32 I2S API has a configurable DMA buffer that is filled when you write data to I2S: I am using a blocking implementation that waits until there is enough space in the buffer to write the data, if the buffer is full.

One variable of optimization is the write size: I am usally using 1024 bytes. The smaller the size, the bigger the overhead but also the bigger the chance that the buffer remains full.

The sound output is generated from this buffer, so the bigger the buffer, the longer the time gaps between the different writes can be for the audio not breaking up. On the negative side: the bigger the buffer, the bigger the lag!

The default setting for the buffer size is rather on the save side to prevent breakups, so you can minimize the lag and memory usage by reducing the buffer size*buffer count.

As soon as the DMA buffer is full, the whole processing is throttled by the I2S data consumption which in your case would be 44100 samples per second.

You can verify this by measuring the thruput as described in this Wiki!

[bjoerngiesler]

That's what I thought -- but that's not what's happening. It's merrily copying its 3072 bytes per cycle for half an hour now, without delays. Here's a bit of the output:

SignalGeneratorStream bytes available: 102400
I2SStream available for write: 3072
Read 3072 bytes from signal generator stream
Wrote 3072 bytes to I2S stream
Player step, time since last step: 15132 microseconds
SignalGeneratorStream bytes available: 102400
I2SStream available for write: 3072
Read 3072 bytes from signal generator stream
Wrote 3072 bytes to I2S stream
Player step, time since last step: 15297 microseconds
SignalGeneratorStream bytes available: 102400
I2SStream available for write: 3072
Read 3072 bytes from signal generator stream
Wrote 3072 bytes to I2S stream

And here's the code:

bool Player::start() {
    pinMode(P_I2S_BCK, OUTPUT);
    pinMode(P_I2S_DATA_OUT, OUTPUT);
    pinMode(P_I2S_WS, OUTPUT);

    auto cfg = i2s_.defaultConfig(TX_MODE);
    cfg.setAudioInfo(info_);
    cfg.pin_data = P_I2S_DATA_OUT;
    cfg.pin_ws = P_I2S_WS;
    cfg.pin_bck = P_I2S_BCK;
    cfg.is_master = true;
    i2s_.begin(cfg);

    sineGen_.begin(info_, N_B5);
    signalGeneratorStream_.setInput(sineGen_);
    signalGeneratorStream_.begin(info_);

    return true;
}
bool Player::step() {
    static unsigned long last_time = micros();
    printf("Player step, time since last step: %lu microseconds\n", micros() - last_time);
    if(micros()-last_time > 20000) {
            printf("*****     More than 20ms since last step!     *****\n");
    }
    last_time = micros();

    printf("SignalGeneratorStream bytes available: %d\n", signalGeneratorStream_.available());
    printf("I2SStream available for write: %d\n", i2s_.availableForWrite());
    int bufsize = min(signalGeneratorStream_.available(), i2s_.availableForWrite());
    if (bufsize < 3072) {
        printf("*****     Less than 3072 bytes available!     *****\n");
        return false;
    }
    uint8_t buffer[bufsize];

    size_t len = signalGeneratorStream_.readBytes(buffer, sizeof(buffer));
    printf("Read %d bytes from signal generator stream\n", (int)len);

    i2s_.write(buffer, len);
    printf("Wrote %d bytes to I2S stream\n", (int)len);

    return true;
}

Player::step() is called from Arduino's loop() which contains that and vTaskDelay(1), nothing else. So it's stating that the whole loop is taking ~15ms, of course I wasn't correct with the 1.7 above; still, 15ms at 44100Hz, 1 channel, 16bit counts 1,323 bytes and not the 3072 it gobbles up. I would expect, as you say, at some point the DMA buffer to be full and either availableForWrite() returning less than its default or i2s_.write() taking significantly longer. That's why I marked those with the asterisk line above. But that never happens. The mind boggles - what am I missing here?

[bjoerngiesler]

Interesting. Your suggestion of using MeasuringStream was great - it reports 88000 bytes per second or (when supplied with the proper AudioInfo) 44000 samples per second. And I have found my error... I was actually not supplying the correct AudioInfo with 1 channel, but the default one with 2 channels. Which gets us into the ballpark of 2650 bytes in 15ms, which is close enough to the 3072 that I'll put it down as measurement noise. Thank you for bearing with me!

[pschatzmann]

The I2S does not provide any functionality to access the DMA buffer fill level, so I am just returning a default value (of 1024 bytes) to make sure that the processing is not stalled. A similar logic applies to the GeneratedSoundStream available() implementation in order to provide a good default to drive the copy size.

The following sketch is using tested functionality and should provide the correct information. Feel free to add some small delays after the copy step in the loop...

#include "AudioTools.h"

AudioInfo info(44100, 1, 16);
SineWaveGenerator<int16_t> sineWave(32000);                // subclass of SoundGenerator with max amplitude of 32000
GeneratedSoundStream<int16_t> sound(sineWave);             // Stream generated from sine wave
I2SStream out;
int reporing_interval = 10;
MeasuringStream measure(out, reporing_interval, &Serial);
StreamCopy copier(measure, sound);                             // copies sound into i2s

// Arduino Setup
void setup(void) {  
  // Open Serial 
  Serial.begin(115200);
  while(!Serial);
  AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Info);

  // start I2S
  Serial.println("starting I2S...");
  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  out.begin(config);

  // setup measuring stream
  measure.begin(info);

  // Setup sine wave
  sineWave.begin(info, N_B4);
  Serial.println("started...");
}

// Arduino loop - copy sound to out 
void loop() {
  copier.copy();
}