AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

payday1991 · 2024-04-04T14:49:01Z

Environment

Audio development kit: [none]
Audio kit version (for ESP32-LyraT/ESP32-LyraT-Mini/ESP32-S3-Korvo-2): [N/A]
[Required] Module or chip used: [ESP32-S3-WROOM-1]
[Required] IDF version (run git describe --tags in $IDF_PATH folder to find it):
v5.1.2
[Required] ADF version (run git describe --tags in $ADF_PATH folder to find it):
v2.6-86-g1a03fe6a
Build system: [idf.py]
[Required] Running log: All logs from power-on to problem recurrence
Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it):
xtensa-esp32s3-elf-gcc.exe (crosstool-NG esp-12.2.0_20230208) 12.2.0
Operating system: [Windows]
(Windows only) Environment type: [ESP Command Prompt]
Using an IDE?: [Yes (VS Code with ESP-IDF extension)]
Power supply: [USB]

Problem Description

Hi.
I'm out of options and desperately need some help.
Maybe somebody had this problem before. Basically, I'm just trying to use the Espressif's Audio Forge example(https://github.com/espressif/esp-adf/blob/master/examples/audio_processing/pipeline_audio_forge/main/audio_forge_pipeline_main.c) from ESP-ADF repository and compile it for my custom ESP32-S3 board. Everything works, and I CAN HEAR sound from the speaker when I

#define NUMBER_SOURCE_FILE 1,

but it's showing me the

"AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set"

when I #define NUMBER_SOURCE_FILE 2.
The modification I've done was to add
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {esp_err_t ret = audio_pipeline_run(pipeline[i]);} audio_pipeline_run(pipeline_mix);
before while (1) {} to start the audio pipeline because I don't have a board with the buttons and use MP3 instead of WAV.

This is my console output:

Expected Behavior

I should hear 2 sounds simultaneously.

Actual Behavior

E (1546) AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set

And silence...

Steps to Reproduce

#define NUMBER_SOURCE_FILE 2
Compile and run.

I (1436) DS1302: [3.0] Create pipeline_mix to mix
I (1436) DS1302: [3.1] Create audio_forge
I (1436) DS1302: [3.2] Create i2s stream to read audio data from codec chip
I (1436) DS1302: [3.3] Link elements together audio_forge-->i2s_writer
I (1436) DS1302: [3.4] Link elements together audio_forge-->i2s_stream-->[codec_chip]
I (1436) DS1302: [4.0] Create Fatfs stream to read input data
I (1436) DS1302: [4.1] Create wav decoder to decode wav file
I (1436) DS1302: [4.2] Create raw stream of base wav to write data
I (1436) DS1302: [5.0] Set up event listener
I (1436) MP3_DECODER: MP3 init
I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c077d78, tag:file, rb:0x3c0781b0
I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c077f10, tag:wav, rb:0x3c07a1f8
I (1436) MP3_DECODER: MP3 init
I (1436) AUDIO_PIPELINE: link el->rb, el:0x3c07aaa8, tag:file, rb:0x3c07aee0
I (1446) gpio: GPIO[0]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1446) gpio: GPIO[1]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:3
I (1446) AUDIO_PIPELINE: link el->rb, el:0x3c07ac40, tag:wav, rb:0x3c07cf28
I (1446) DS1302: [5.1] Listening event from peripherals
I (1446) AUDIO_THREAD: The file task allocate stack on internal memory
I (1446) AUDIO_ELEMENT: [file-0x3c077d78] Element task created
I (1446) AUDIO_THREAD: The wav task allocate stack on external memory
I (1446) AUDIO_ELEMENT: [wav-0x3c077f10] Element task created
I (1446) AUDIO_ELEMENT: [raw-0x3c078040] Element task created
I (1446) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8634896 Bytes, Inter:340923 Bytes, Dram:340923 Bytes

I (1446) AUDIO_ELEMENT: [file] AEL_MSG_CMD_RESUME,state:1
I (1446) AUDIO_ELEMENT: [wav] AEL_MSG_CMD_RESUME,state:1
I (1446) MP3_DECODER: MP3 opened
I (1446) FATFS_STREAM: File size: 767885 byte, file position: 0
I (1446) AUDIO_PIPELINE: Pipeline started
I (1446) AUDIO_THREAD: The file task allocate stack on internal memory
I (1446) AUDIO_ELEMENT: [file-0x3c07aaa8] Element task created
I (1446) AUDIO_THREAD: The wav task allocate stack on external memory
I (1456) AUDIO_ELEMENT: [wav-0x3c07ac40] Element task created
I (1456) AUDIO_ELEMENT: [raw-0x3c07ad70] Element task created
I (1456) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8618684 Bytes, Inter:335531 Bytes, Dram:335531 Bytes

I (1456) AUDIO_ELEMENT: [file] AEL_MSG_CMD_RESUME,state:1
I (1456) AUDIO_ELEMENT: [wav] AEL_MSG_CMD_RESUME,state:1
I (1456) MP3_DECODER: MP3 opened
I (1456) AUDIO_PIPELINE: Pipeline started
I (1456) AUDIO_THREAD: The audio_forge task allocate stack on external memory
I (1456) CODEC_ELEMENT_HELPER: The element is 0x3c077f10. The reserve data 2 is 0x0.
I (1466) AUDIO_ELEMENT: [audio_forge-0x3c0775e8] Element task created
I (1466) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8575400 Bytes, Inter:335703 Bytes, Dram:335703 Bytes

I (1466) AUDIO_ELEMENT: [audio_forge] AEL_MSG_CMD_RESUME,state:1
I (1466) AUDIO_FORGE: audio_forge opened
I (1466) AUDIO_PIPELINE: Pipeline started
W (1466) DS1302: [ * ] Receive music info from wav decoder, sample_rates=44100, bits=16, ch=1
I (1476) FATFS_STREAM: File size: 736584 byte, file position: 0
I (1486) CODEC_ELEMENT_HELPER: The element is 0x3c07ac40. The reserve data 2 is 0x0.
W (1526) DS1302: [ * ] Receive music info from wav decoder, sample_rates=44100, bits=16, ch=1
E (1546) AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set
I (1546) AUDIO_FORGE: audio forge closed
I (1546) DS1302: [6.0] Stop pipelines
W (1546) AUDIO_ELEMENT: OUT-[wav] AEL_IO_ABORT
W (1546) MP3_DECODER: Output aborted -3
I (1546) MP3_DECODER: Closed
W (1546) AUDIO_ELEMENT: OUT-[file] AEL_IO_ABORT
W (1556) AUDIO_ELEMENT: OUT-[wav] AEL_IO_ABORT
W (1556) MP3_DECODER: Output aborted -3
I (1556) MP3_DECODER: Closed
W (1556) AUDIO_ELEMENT: OUT-[file] AEL_IO_ABORT
W (1556) AUDIO_ELEMENT: [audio_forge] Element already stopped
W (1556) AUDIO_PIPELINE: There are no listener registered
I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked
W (1556) AUDIO_ELEMENT: [file] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_ELEMENT: [wav] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_ELEMENT: [raw] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_PIPELINE: There are no listener registered
I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked
W (1556) AUDIO_ELEMENT: [file] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_ELEMENT: [wav] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_ELEMENT: [raw] Element has not create when AUDIO_ELEMENT_TERMINATE
W (1556) AUDIO_PIPELINE: There are no listener registered
I (1556) AUDIO_PIPELINE: audio_pipeline_unlinked
W (1556) AUDIO_ELEMENT: [audio_forge] Element has not create when AUDIO_ELEMENT_TERMINATE
I (1556) AUDIO_FORGE: audio_forge_destroy
I (1556) AUDIO_FORGE: Func:audio_forge_destroy, Line:406, MEM Total:8674748 Bytes, Inter:349959 Bytes, Dram:349959 Bytes

W (1556) AUDIO_ELEMENT: [iis] Element has not create when AUDIO_ELEMENT_TERMINATE


## Other Items If Possible

- [ ] sdkconfig file (Attach the sdkconfig file from your project folder)
[sdkconfig.txt](https://github.com/espressif/esp-adf/files/14872947/sdkconfig.txt)

The text was updated successfully, but these errors were encountered:

jason-mao · 2024-04-07T03:52:09Z

@payday1991 Thanks for your report. Let's check and reply.

jason-mao · 2024-04-08T01:10:02Z

It's fixed on b3add92

majingjing123 · 2024-04-08T02:30:23Z

Hi, this issue is already fixed. If you only need mix, you can use example audio_mixer_tone.
If you must use audio forege, for your needs, you can use this test code
`
/* Multiple pipeline playback with audio processing.

This example code is in the Public Domain (or CC0 licensed, at your option.)

Unless required by applicable law or agreed to in writing, this
software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied.
*/

#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
#include "esp_log.h"
#include "audio_element.h"
#include "audio_pipeline.h"
#include "audio_event_iface.h"
#include "i2s_stream.h"
#include "mp3_decoder.h"
#include "fatfs_stream.h"
#include "audio_forge.h"
#include "raw_stream.h"
#include "board.h"
#include "periph_button.h"

static const char *TAG = "AUDIO_FORGE_PIPELINE";

#define DEFAULT_SAMPLERATE 44100
#define DEFAULT_CHANNEL 2
#define DEST_SAMPLERATE 11025
#define DEST_CHANNEL 1
#define TRANSMITTIME 0
#define MUSIC_GAIN_DB -10
#define NUMBER_SOURCE_FILE 2

int audio_forge_wr_cb(audio_element_handle_t el, char *buf, int len, TickType_t wait_time, void *ctx)
{
audio_element_handle_t i2s_wr = (audio_element_handle_t)ctx;
int ret = audio_element_output(i2s_wr, buf, len);
return ret;
}

void app_main(void)
{
audio_pipeline_handle_t pipeline[NUMBER_SOURCE_FILE] = {NULL};
audio_element_handle_t fats_rd_el[NUMBER_SOURCE_FILE] = {NULL};
audio_element_handle_t mp3_decoder[NUMBER_SOURCE_FILE] = {NULL};
audio_element_handle_t el_raw_write[NUMBER_SOURCE_FILE] = {NULL};

esp_log_level_set("*", ESP_LOG_INFO);
esp_log_level_set(TAG, ESP_LOG_INFO);

ESP_LOGI(TAG, "[1.0] Start audio codec chip");
audio_board_handle_t board_handle = audio_board_init();
audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_DECODE, AUDIO_HAL_CTRL_START);

ESP_LOGI(TAG, "[2.0] Start and wait for SDCARD to mount");
esp_periph_config_t periph_cfg = DEFAULT_ESP_PERIPH_SET_CONFIG();
esp_periph_set_handle_t set = esp_periph_set_init(&periph_cfg);
audio_board_sdcard_init(set, SD_MODE_1_LINE);
audio_board_key_init(set);

ESP_LOGI(TAG, "[3.0] Create pipeline_mix to mix");
audio_pipeline_cfg_t pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG();
audio_pipeline_handle_t pipeline_mix = audio_pipeline_init(&pipeline_cfg);

ESP_LOGI(TAG, "[3.1] Create audio_forge");
audio_forge_cfg_t audio_forge_cfg = AUDIO_FORGE_CFG_DEFAULT();
audio_forge_cfg.audio_forge.component_select = AUDIO_FORGE_SELECT_RESAMPLE | AUDIO_FORGE_SELECT_DOWNMIX | AUDIO_FORGE_SELECT_ALC | AUDIO_FORGE_SELECT_EQUALIZER | AUDIO_FORGE_SELECT_SONIC;
audio_forge_cfg.audio_forge.dest_samplerate = DEST_SAMPLERATE;
audio_forge_cfg.audio_forge.dest_channel = DEST_CHANNEL;
audio_forge_cfg.audio_forge.source_num = NUMBER_SOURCE_FILE;
audio_forge_cfg.audio_forge.max_sample = 128;
audio_forge_cfg.task_prio = 4;
audio_element_handle_t audio_forge = audio_forge_init(&audio_forge_cfg);
audio_forge_src_info_t source_information = {
    .samplerate = DEFAULT_SAMPLERATE,
    .channel = DEFAULT_CHANNEL,
    .bit_num = 16,
};

audio_forge_downmix_t downmix_information = {
    .gain = {0, MUSIC_GAIN_DB},
    .transit_time = TRANSMITTIME,
};
audio_forge_src_info_t source_info[NUMBER_SOURCE_FILE] = {0};
audio_forge_downmix_t downmix_info[NUMBER_SOURCE_FILE];
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    source_info[i] = source_information;
    downmix_info[i] = downmix_information;
}
audio_forge_source_info_init(audio_forge, source_info, downmix_info);

ESP_LOGI(TAG, "[3.2] Create i2s stream to read audio data from codec chip");
i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
i2s_cfg.type = AUDIO_STREAM_WRITER;
i2s_cfg.task_stack = 0;
i2s_cfg.out_rb_size = 0;

#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)
i2s_cfg.chan_cfg.auto_clear = true;
#else
i2s_cfg.i2s_config.tx_desc_auto_clear = true;
#endif
audio_element_handle_t i2s_writer = i2s_stream_init(&i2s_cfg);
i2s_stream_set_clk(i2s_writer, DEST_SAMPLERATE, 16, DEST_CHANNEL);

ESP_LOGI(TAG, "[3.3] Link elements together audio_forge-->i2s_writer");
audio_pipeline_register(pipeline_mix, audio_forge, "audio_forge");
audio_element_set_write_cb(audio_forge, audio_forge_wr_cb, i2s_writer);
audio_element_process_init(i2s_writer);


ESP_LOGI(TAG, "[3.4] Link elements together audio_forge-->i2s_stream-->[codec_chip]");
audio_pipeline_link(pipeline_mix, (const char *[]) {"audio_forge"}, 1);

ESP_LOGI(TAG, "[4.0] Create Fatfs stream to read input data");
fatfs_stream_cfg_t fatfs_cfg = FATFS_STREAM_CFG_DEFAULT();
fatfs_cfg.type = AUDIO_STREAM_READER;

ESP_LOGI(TAG, "[4.1] Create mp3 decoder to decode mp3 file");
mp3_decoder_cfg_t mp3_cfg = DEFAULT_MP3_DECODER_CONFIG();
mp3_cfg.task_core = 0;

ESP_LOGI(TAG, "[4.2] Create raw stream of base mp3 to write data");
raw_stream_cfg_t raw_cfg = RAW_STREAM_CFG_DEFAULT();
raw_cfg.type = AUDIO_STREAM_WRITER;

ESP_LOGI(TAG, "[5.0] Set up  event listener");
audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
audio_event_iface_handle_t evt = audio_event_iface_init(&evt_cfg);

char str_name[18] = "/sdcard/test";
char num = '1';
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    pipeline[i] = audio_pipeline_init(&pipeline_cfg);
    mem_assert(pipeline[i]);
    fats_rd_el[i] = fatfs_stream_init(&fatfs_cfg);
    str_name[12] = num + i;
    str_name[13] = '.';
    str_name[14] = 'm';
    str_name[15] = 'p';
    str_name[16] = '3';
    audio_element_set_uri(fats_rd_el[i], str_name);
    mp3_cfg.task_core = i; // mp3 decoder set to differen cpu core
    // mp3_cfg.task_prio = 12 - i - i;
    mp3_decoder[i] = mp3_decoder_init(&mp3_cfg);
    el_raw_write[i] = raw_stream_init(&raw_cfg);
    audio_pipeline_register(pipeline[i], fats_rd_el[i], "file");
    audio_pipeline_register(pipeline[i], mp3_decoder[i], "mp3");
    audio_pipeline_register(pipeline[i], el_raw_write[i], "raw");
      
    const char *link_tag[3] = {"file", "mp3", "raw"};
    audio_pipeline_link(pipeline[i], &link_tag[0], 3);
    ringbuf_handle_t rb = audio_element_get_input_ringbuf(el_raw_write[i]);
    if (NUMBER_SOURCE_FILE != 1) {
        audio_element_set_multi_input_ringbuf(audio_forge, rb, i);
    } else {
        audio_element_set_input_ringbuf(audio_forge, rb);
    }
    audio_pipeline_set_listener(pipeline[i], evt);
}
audio_pipeline_set_listener(pipeline_mix, evt);
ESP_LOGI(TAG, "[5.1] Listening event from peripherals");
audio_event_iface_set_listener(esp_periph_set_get_event_iface(set), evt);
audio_forge_downmix_set_transit_time(audio_forge, 100, 0);
audio_forge_downmix_set_transit_time(audio_forge, 100, 1);
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    audio_pipeline_run(pipeline[i]);
}
audio_pipeline_run(pipeline_mix);
while (1) {
    audio_event_iface_msg_t msg;
    esp_err_t ret = audio_event_iface_listen(evt, &msg, portMAX_DELAY);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "[ * ] Event interface error : %d", ret);
        continue;
    }

    for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT && msg.source == (void *)mp3_decoder[i]
            && msg.cmd == AEL_MSG_CMD_REPORT_MUSIC_INFO) {
            audio_element_info_t music_info = {0};
            audio_element_getinfo(mp3_decoder[i], &music_info);
            ESP_LOGW(TAG, "[ * ] Receive music info from wav decoder, sample_rates=%d, bits=%d, ch=%d",
                     music_info.sample_rates, music_info.bits, music_info.channels);
            audio_forge_src_info_t src_info = {
                .samplerate = music_info.sample_rates,
                .channel = music_info.channels,
                .bit_num = music_info.bits,
            };
            audio_forge_set_src_info(audio_forge, src_info, i);
        }
    }
    /* Stop when the last pipeline element (fatfs_writer in this case) receives stop event */
    if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT
        && msg.source == (void *) audio_forge
        && msg.cmd == AEL_MSG_CMD_REPORT_STATUS
        && (((int)msg.data == AEL_STATUS_STATE_STOPPED)
            || ((int)msg.data == AEL_STATUS_STATE_FINISHED))) {
        break;
    }
}

ESP_LOGI(TAG, "[6.0] Stop pipelines");
for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    audio_pipeline_stop(pipeline[i]);
    audio_pipeline_wait_for_stop(pipeline[i]);
    audio_pipeline_terminate(pipeline[i]);
    audio_pipeline_unregister_more(pipeline[i], fats_rd_el[i], mp3_decoder[i], el_raw_write[i], NULL);
    audio_pipeline_remove_listener(pipeline[i]);
}
audio_pipeline_stop(pipeline_mix);
audio_pipeline_wait_for_stop(pipeline_mix);
audio_pipeline_terminate(pipeline_mix);
audio_pipeline_unregister_more(pipeline_mix, audio_forge, NULL);
audio_pipeline_remove_listener(pipeline_mix);

/* Stop all peripherals before removing the listener */
esp_periph_set_stop_all(set);
audio_event_iface_remove_listener(esp_periph_set_get_event_iface(set), evt);

/* Make sure audio_pipeline_remove_listener & audio_event_iface_remove_listener are called before destroying event_iface */
audio_event_iface_destroy(evt);

for (int i = 0; i < NUMBER_SOURCE_FILE; i++) {
    /* Release resources */
    audio_pipeline_deinit(pipeline[i]);
    audio_element_deinit(fats_rd_el[i]);
    audio_element_deinit(mp3_decoder[i]);
    audio_element_deinit(el_raw_write[i]);
}
/* Release resources */
audio_pipeline_deinit(pipeline_mix);
audio_element_deinit(audio_forge);
audio_element_deinit(i2s_writer);
esp_periph_set_destroy(set);

}
`

payday1991 · 2024-04-08T06:30:52Z

Thank you.

jason-mao closed this as completed Apr 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

payday1991 commented Apr 4, 2024 •

edited

Loading

jason-mao commented Apr 7, 2024

jason-mao commented Apr 8, 2024

majingjing123 commented Apr 8, 2024

payday1991 commented Apr 8, 2024

AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

AUDIO_ELEMENT: [audio_forge] Read IO type ringbuf but ringbuf not set when #define NUMBER_SOURCE_FILE 2 in Audio Forge Example. (AUD-5319) #1186

Comments

payday1991 commented Apr 4, 2024 • edited Loading

Environment

Problem Description

Expected Behavior

Actual Behavior

Steps to Reproduce

jason-mao commented Apr 7, 2024

jason-mao commented Apr 8, 2024

majingjing123 commented Apr 8, 2024

payday1991 commented Apr 8, 2024

payday1991 commented Apr 4, 2024 •

edited

Loading