Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic ESP32-S3 TinyUSB Audio Device Example? (IDFGH-11661) #12774

Open
akumpf opened this issue Dec 11, 2023 · 11 comments
Open

Basic ESP32-S3 TinyUSB Audio Device Example? (IDFGH-11661) #12774

akumpf opened this issue Dec 11, 2023 · 11 comments
Assignees
Labels
Status: Opened Issue is new Type: Feature Request Feature request for IDF

Comments

@akumpf
Copy link

akumpf commented Dec 11, 2023

Is your feature request related to a problem?

Being able to route lossless audio to/from a computer is super helpful for debugging a very wide range of audio projects (audio effects, speech, AI, synthesizers, etc.).

The underlying TinyUSB library includes support for UAC2 audio, and ESP32-S2/S3 should be able to handle this just fine. But it doesn't look like the ESP-IDF + esp_tinyusb component supports it yet. What would it take to port over the basic TinyUSB "audio_test" example? Or should esp_tinyusb not be used at all, and just directly use tinyusb?

Describe the solution you'd like.

So far, it looks like we'll need:

  • a Kconfig option to set the CFG_TUD_AUDIO flag to 1
  • pulling in the relevant descriptors from the example.

For the example, I think it could be limited to a single audio channel (like the "one channel mic" used in the example) at a fixed samplerate (like 48000).

Describe alternatives you've considered.

I've been able to compile the TinyUSB audio_test example and upload to an ESP32-S3, but the USB audio interface doesn't show up. I'm guessing it has to do with some specific low-level knowledge of USB descriptors/limitations, so hoping that someone more familiar with USB it may be able to highlight the important flags/settings to get things up and running with USB audio.

Additional context.

No response

@akumpf akumpf added the Type: Feature Request Feature request for IDF label Dec 11, 2023
@espressif-bot espressif-bot added the Status: Opened Issue is new label Dec 11, 2023
@github-actions github-actions bot changed the title Basic ESP32-S3 TinyUSB Audio Device Example? Basic ESP32-S3 TinyUSB Audio Device Example? (IDFGH-11661) Dec 11, 2023
@akumpf
Copy link
Author

akumpf commented Dec 11, 2023

For completeness, here's the code I've been using to test USB UAC2 audio. As mentioned, it compiles/uploads fine but doesn't actually show us as a USB audio device on the host computer.

// ----------------------------------------------
static const char *TAG_USB = "usb";
// ----------------------------------------------
// Force AUDIO since no flag currently in Kconfig/menuconfig...
#define CFG_TUD_AUDIO   1 
// ----------------------------------------------
// see: https://github.com/hathach/tinyusb/blob/master/examples/device/audio_test/src/tusb_config.h
#define CFG_TUD_AUDIO_FUNC_1_DESC_LEN                                 TUD_AUDIO_MIC_ONE_CH_DESC_LEN
#define CFG_TUD_AUDIO_FUNC_1_N_AS_INT                                 1                                       // Number of Standard AS Interface Descriptors (4.9.1) defined per audio function - this is required to be able to remember the current alternate settings of these interfaces - We restrict us here to have a constant number for all audio functions (which means this has to be the maximum number of AS interfaces an audio function has and a second audio function with less AS interfaces just wastes a few bytes)
#define CFG_TUD_AUDIO_FUNC_1_CTRL_BUF_SZ                              64                                      // Size of control request buffer
#define CFG_TUD_AUDIO_ENABLE_EP_IN                                    1
#define CFG_TUD_AUDIO_FUNC_1_N_BYTES_PER_SAMPLE_TX                    2                                       // Driver gets this info from the descriptors - we define it here to use it to setup the descriptors and to do calculations with it below
#define CFG_TUD_AUDIO_FUNC_1_N_CHANNELS_TX                            1                                       // Driver gets this info from the descriptors - we define it here to use it to setup the descriptors and to do calculations with it below - be aware: for different number of channels you need another descriptor!
#define CFG_TUD_AUDIO_EP_SZ_IN                                        (48+1) * CFG_TUD_AUDIO_FUNC_1_N_BYTES_PER_SAMPLE_TX * CFG_TUD_AUDIO_FUNC_1_N_CHANNELS_TX      // 48 Samples (48 kHz) x 2 Bytes/Sample x CFG_TUD_AUDIO_N_CHANNELS_TX Channels - One extra sample is needed for asynchronous transfer adjustment, see feedback EP
#define CFG_TUD_AUDIO_FUNC_1_EP_IN_SZ_MAX                             CFG_TUD_AUDIO_EP_SZ_IN                  // Maximum EP IN size for all AS alternate settings used
#define CFG_TUD_AUDIO_FUNC_1_EP_IN_SW_BUF_SZ                          CFG_TUD_AUDIO_EP_SZ_IN + 1
// ---- 
#include <stdlib.h>
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_timer.h"
#include "tinyusb.h"
// ---- 
// Interface counter
enum usb_interface_count {
  ITF_NUM_AUDIO_CONTROL = 0,
  ITF_NUM_AUDIO_STREAMING_MIC,
  ITF_COUNT
};
// USB Endpoint numbers: Available USB Endpoints: 5 IN/OUT EPs and 1 IN EP
#define USB_EPNUM_AUDIO     0x01

//--------------------------------------------------------------------+
// Device Descriptors
//--------------------------------------------------------------------+
#define _PID_MAP(itf, n)  ( (CFG_TUD_##itf) << (n) )
#define USB_PID           (0x4000 | _PID_MAP(CDC, 0) | _PID_MAP(MSC, 1) | _PID_MAP(HID, 2) | _PID_MAP(MIDI, 3) | _PID_MAP(AUDIO, 4) | _PID_MAP(VENDOR, 5) )
tusb_desc_device_t const usb_device_desc ={
    .bLength            = sizeof(tusb_desc_device_t),
    .bDescriptorType    = TUSB_DESC_DEVICE,
    .bcdUSB             = 0x0200,
    // Use Interface Association Descriptor (IAD) for Audio
    // As required by USB Specs IAD's subclass must be common class (2) and protocol must be IAD (1)
    .bDeviceClass       = TUSB_CLASS_MISC,
    .bDeviceSubClass    = MISC_SUBCLASS_COMMON,
    .bDeviceProtocol    = MISC_PROTOCOL_IAD,
    .bMaxPacketSize0    = CFG_TUD_ENDPOINT0_SIZE,
    .idVendor           = 0xCafe,
    .idProduct          = USB_PID,
    .bcdDevice          = 0x0100,
    .iManufacturer      = 0x01,
    .iProduct           = 0x02,
    .iSerialNumber      = 0x03,
    .bNumConfigurations = 0x01
};
// TinyUSB descriptors
#define TUSB_DESCRIPTOR_TOTAL_LEN (TUD_CONFIG_DESC_LEN + CFG_TUD_AUDIO * TUD_AUDIO_MIC_ONE_CH_DESC_LEN)

// String descriptor
static const char* usb_string_desc_arr[7] = {
    // array of pointer to string descriptors
    (char[]){0x09, 0x04},   // 0: is supported language is English (0x0409)
    "TinyUSB",              // 1: Manufacturer
    "Product",              // 2: Product
    "12345678",             // 3: Serials, should use chip ID
    "AudioName",            // 4: Audio
};
// --
// Configuration descriptor
static const uint8_t usb_desc_configuration[] = {
  // Configuration number, interface count, string index, total length, attribute, power in mA
  TUD_CONFIG_DESCRIPTOR(1, ITF_COUNT, 0, TUSB_DESCRIPTOR_TOTAL_LEN, 0x00, 100),

  // Audio Mic: Interface number, string index, EP Out & EP In address, EP size
  // Verified that TUD_AUDIO_MIC_ONE_CH_DESC_LEN = 132 = length of result for 'TUD_AUDIO_MIC_ONE_CH_DESCRIPTOR()'. So length is not the issue.
  TUD_AUDIO_MIC_ONE_CH_DESCRIPTOR(ITF_NUM_AUDIO_CONTROL, 4, CFG_TUD_AUDIO_FUNC_1_N_BYTES_PER_SAMPLE_TX, CFG_TUD_AUDIO_FUNC_1_N_BYTES_PER_SAMPLE_TX*8, 0x80 | USB_EPNUM_AUDIO, CFG_TUD_AUDIO_EP_SZ_IN),
};
//--------------------------------------------------------------------+
// USB Init
//--------------------------------------------------------------------+
void usb_init(){
  tinyusb_config_t const tusb_cfg = {
      .device_descriptor = &usb_device_desc, // If device_descriptor is NULL, tinyusb_driver_install() will use Kconfig
      .string_descriptor = usb_string_desc_arr,
      .string_descriptor_count = sizeof(usb_string_desc_arr) / sizeof(usb_string_desc_arr[0]),
      .external_phy = false, // false = use internal USB interface (i.e. not an external USB chip)
      .configuration_descriptor = usb_desc_configuration,
  };
  ESP_ERROR_CHECK(tinyusb_driver_install(&tusb_cfg));
  ESP_LOGI(TAG_USB, "USB initialization DONE");
}

void app_main(void){
  usb_init();
  // --
  while(1){
    ESP_LOGI(TAG_USB, "testing usb audio");
    vTaskDelay(1000 / portTICK_PERIOD_MS); // Handle for low-speed updates at ~100Hz (10ms).
  }
}


//--------------------------------------------------------------------+
// Audio STATE
//--------------------------------------------------------------------+

// Audio controls
// Current states
bool mute[CFG_TUD_AUDIO_FUNC_1_N_CHANNELS_TX + 1]; 				          // +1 for master channel 0
uint16_t volume[CFG_TUD_AUDIO_FUNC_1_N_CHANNELS_TX + 1]; 					// +1 for master channel 0
uint32_t sampFreq;
uint8_t clkValid;

// Range states
audio_control_range_2_n_t(1) volumeRng[CFG_TUD_AUDIO_FUNC_1_N_CHANNELS_TX+1]; 			// Volume range state
audio_control_range_4_n_t(1) sampleFreqRng; 						// Sample frequency range state

// Audio test data
uint16_t test_buffer_audio[(CFG_TUD_AUDIO_EP_SZ_IN - 2) / 2];
uint16_t startVal = 0;

//--------------------------------------------------------------------+
// Device callbacks
//--------------------------------------------------------------------+
// Invoked when device is mounted
void tud_mount_cb(void){
//   blink_interval_ms = BLINK_MOUNTED;
}

// Invoked when device is unmounted
void tud_umount_cb(void){
//   blink_interval_ms = BLINK_NOT_MOUNTED;
}

// Invoked when usb bus is suspended
// remote_wakeup_en : if host allow us  to perform remote wakeup
// Within 7ms, device must draw an average of current less than 2.5 mA from bus
void tud_suspend_cb(bool remote_wakeup_en){
  (void) remote_wakeup_en;
//   blink_interval_ms = BLINK_SUSPENDED;
}

// Invoked when usb bus is resumed
void tud_resume_cb(void){
//   blink_interval_ms = tud_mounted() ? BLINK_MOUNTED : BLINK_NOT_MOUNTED;
}


//--------------------------------------------------------------------+
// Application Callback API Implementations
//--------------------------------------------------------------------+
// Invoked when audio class specific set request received for an EP
bool tud_audio_set_req_ep_cb(uint8_t rhport, tusb_control_request_t const * p_request, uint8_t *pBuff){
  (void) rhport;
  (void) pBuff;
  // We do not support any set range requests here, only current value requests
  TU_VERIFY(p_request->bRequest == AUDIO_CS_REQ_CUR);
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  uint8_t ep = TU_U16_LOW(p_request->wIndex);
  (void) channelNum; (void) ctrlSel; (void) ep;
  return false; 	// Yet not implemented
}

// Invoked when audio class specific set request received for an interface
bool tud_audio_set_req_itf_cb(uint8_t rhport, tusb_control_request_t const * p_request, uint8_t *pBuff){
  (void) rhport;
  (void) pBuff;
  // We do not support any set range requests here, only current value requests
  TU_VERIFY(p_request->bRequest == AUDIO_CS_REQ_CUR);
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  uint8_t itf = TU_U16_LOW(p_request->wIndex);
  (void) channelNum; (void) ctrlSel; (void) itf;
  return false; 	// Yet not implemented
}

// Invoked when audio class specific set request received for an entity
bool tud_audio_set_req_entity_cb(uint8_t rhport, tusb_control_request_t const * p_request, uint8_t *pBuff){
  (void) rhport;
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  uint8_t itf = TU_U16_LOW(p_request->wIndex);
  uint8_t entityID = TU_U16_HIGH(p_request->wIndex);
  (void) itf;
  // We do not support any set range requests here, only current value requests
  TU_VERIFY(p_request->bRequest == AUDIO_CS_REQ_CUR);
  // If request is for our feature unit
  if ( entityID == 2 ){
    switch ( ctrlSel ){
      case AUDIO_FU_CTRL_MUTE:
        // Request uses format layout 1
        TU_VERIFY(p_request->wLength == sizeof(audio_control_cur_1_t));
        mute[channelNum] = ((audio_control_cur_1_t*) pBuff)->bCur;
        TU_LOG2("    Set Mute: %d of channel: %u\r\n", mute[channelNum], channelNum);
      return true;
      // --
      case AUDIO_FU_CTRL_VOLUME:
        // Request uses format layout 2
        TU_VERIFY(p_request->wLength == sizeof(audio_control_cur_2_t));
        volume[channelNum] = (uint16_t) ((audio_control_cur_2_t*) pBuff)->bCur;
        TU_LOG2("    Set Volume: %d dB of channel: %u\r\n", volume[channelNum], channelNum);
      return true;
        // Unknown/Unsupported control
      default:
        TU_BREAKPOINT();
      return false;
    }
  }
  return false;    // Yet not implemented
}
// Invoked when audio class specific get request received for an EP
bool tud_audio_get_req_ep_cb(uint8_t rhport, tusb_control_request_t const * p_request){
  (void) rhport;
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  uint8_t ep = TU_U16_LOW(p_request->wIndex);
  (void) channelNum; (void) ctrlSel; (void) ep;
  //	return tud_control_xfer(rhport, p_request, &tmp, 1);
  return false; 	// Yet not implemented
}

// Invoked when audio class specific get request received for an interface
bool tud_audio_get_req_itf_cb(uint8_t rhport, tusb_control_request_t const * p_request){
  (void) rhport;
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  uint8_t itf = TU_U16_LOW(p_request->wIndex);
  (void) channelNum; (void) ctrlSel; (void) itf;
  return false; 	// Yet not implemented
}

// Invoked when audio class specific get request received for an entity
bool tud_audio_get_req_entity_cb(uint8_t rhport, tusb_control_request_t const * p_request){
  (void) rhport;
  // Page 91 in UAC2 specification
  uint8_t channelNum = TU_U16_LOW(p_request->wValue);
  uint8_t ctrlSel = TU_U16_HIGH(p_request->wValue);
  // uint8_t itf = TU_U16_LOW(p_request->wIndex); 			// Since we have only one audio function implemented, we do not need the itf value
  uint8_t entityID = TU_U16_HIGH(p_request->wIndex);
  // Input terminal (Microphone input)
  if (entityID == 1){
    switch ( ctrlSel ){
      case AUDIO_TE_CTRL_CONNECTOR:{
        // The terminal connector control only has a get request with only the CUR attribute.
        audio_desc_channel_cluster_t ret;
        // Those are dummy values for now
        ret.bNrChannels = 1;
        ret.bmChannelConfig = (audio_channel_config_t) 0;
        ret.iChannelNames = 0;
        TU_LOG2("    Get terminal connector\r\n");
        return tud_audio_buffer_and_schedule_control_xfer(rhport, p_request, (void*) &ret, sizeof(ret));
      }
      break;
        // Unknown/Unsupported control selector
      default:
        TU_BREAKPOINT();
        return false;
    }
  }
  // Feature unit
  if (entityID == 2){
    switch ( ctrlSel ){
      case AUDIO_FU_CTRL_MUTE:
        // Audio control mute cur parameter block consists of only one byte - we thus can send it right away
        // There does not exist a range parameter block for mute
        TU_LOG2("    Get Mute of channel: %u\r\n", channelNum);
        return tud_control_xfer(rhport, p_request, &mute[channelNum], 1);
      case AUDIO_FU_CTRL_VOLUME:
        switch ( p_request->bRequest ){
          case AUDIO_CS_REQ_CUR:
            TU_LOG2("    Get Volume of channel: %u\r\n", channelNum);
            return tud_control_xfer(rhport, p_request, &volume[channelNum], sizeof(volume[channelNum]));
          case AUDIO_CS_REQ_RANGE:
            TU_LOG2("    Get Volume range of channel: %u\r\n", channelNum);
            // Copy values - only for testing - better is version below
            audio_control_range_2_n_t(1)
            ret;
            ret.wNumSubRanges = 1;
            ret.subrange[0].bMin = -90;    // -90 dB
            ret.subrange[0].bMax = 90;		// +90 dB
            ret.subrange[0].bRes = 1; 		// 1 dB steps
            return tud_audio_buffer_and_schedule_control_xfer(rhport, p_request, (void*) &ret, sizeof(ret));
            // Unknown/Unsupported control
          default:
            TU_BREAKPOINT();
            return false;
        }
      break;
        // Unknown/Unsupported control
      default:
        TU_BREAKPOINT();
        return false;
    }
  }
  // Clock Source unit
  if ( entityID == 4 ){
    switch ( ctrlSel ){
      case AUDIO_CS_CTRL_SAM_FREQ:
        // channelNum is always zero in this case
        switch ( p_request->bRequest ){
          case AUDIO_CS_REQ_CUR:
            TU_LOG2("    Get Sample Freq.\r\n");
            return tud_control_xfer(rhport, p_request, &sampFreq, sizeof(sampFreq));
          case AUDIO_CS_REQ_RANGE:
            TU_LOG2("    Get Sample Freq. range\r\n");
            return tud_control_xfer(rhport, p_request, &sampleFreqRng, sizeof(sampleFreqRng));
          // Unknown/Unsupported control
          default:
            TU_BREAKPOINT();
            return false;
        }
      break;
      case AUDIO_CS_CTRL_CLK_VALID:
        // Only cur attribute exists for this request
        TU_LOG2("    Get Sample Freq. valid\r\n");
        return tud_control_xfer(rhport, p_request, &clkValid, sizeof(clkValid));
      // Unknown/Unsupported control
      default:
        TU_BREAKPOINT();
        return false;
    }
  }
  TU_LOG2("  Unsupported entity: %d\r\n", entityID);
  return false; 	// Yet not implemented
}
bool tud_audio_tx_done_pre_load_cb(uint8_t rhport, uint8_t itf, uint8_t ep_in, uint8_t cur_alt_setting){
  (void) rhport;
  (void) itf;
  (void) ep_in;
  (void) cur_alt_setting;
  tud_audio_write ((uint8_t *)test_buffer_audio, CFG_TUD_AUDIO_EP_SZ_IN - 2);
  return true;
}
bool tud_audio_tx_done_post_load_cb(uint8_t rhport, uint16_t n_bytes_copied, uint8_t itf, uint8_t ep_in, uint8_t cur_alt_setting){
  (void) rhport;
  (void) n_bytes_copied;
  (void) itf;
  (void) ep_in;
  (void) cur_alt_setting;
  for (size_t cnt = 0; cnt < (CFG_TUD_AUDIO_EP_SZ_IN - 2) / 2; cnt++){
    test_buffer_audio[cnt] = startVal++;
  }
  return true;
}
bool tud_audio_set_itf_close_EP_cb(uint8_t rhport, tusb_control_request_t const * p_request){
  (void) rhport;
  (void) p_request;
  startVal = 0;
  return true;
}

@espressif-bot espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Dec 12, 2023
@roma-jam
Copy link
Collaborator

Hi @akumpf ,
Thanks for the reported fully described problem.

ESP-IDF uses the esp_tinyusb component which in it's own turn uses the tinyUSB component.
Defining the CFG_TUD_AUDIO works only for the first esp_tinyusb extra component (https://components.espressif.com/components/espressif/esp_tinyusb), but right now there is no support of audio device.
Apparently, the define doesn't works for tinyUSB and this value will be redefined later.

That is why you don't see the device, despite the fact that example has been written correctly.

Anyway, the idea to make this possible can be treated as a good addition to the esp_tinyusb idf-extra-component. We add that feature to the plan.

For now there is a two options available:

It is hard to say any particular date right now, as currently we are busy with adding HighSpeed support for upcoming chips.
Anyway, pull-requests always appreciated.

@espressif-bot espressif-bot added Status: Opened Issue is new and removed Status: In Progress Work is in progress labels Dec 12, 2023
@akumpf
Copy link
Author

akumpf commented Dec 12, 2023

Thanks for the quick follow-up @roma-jam.

Really looking forward to AUDIO being part of esp_tinyusb, but for now I can at least try to play/explore a bit using the TinyUSB component directly. 🤞 😅

@roma-jam
Copy link
Collaborator

Hi @akumpf,
No problem at all. 👍

I'm guessing it has to do with some specific low-level knowledge of USB descriptors/limitations

Actually, no. All the descriptors within your example provided correctly. The sources of audio driver in tinyUSB is compiled, so only configuration CFG_TUD_AUDIO is missing. So the changes to make the example work are pretty simple. On the other hand, to make the configuration universal - this might require a little bit more time.

Meanwhile, do you have anything extra that you would like to have as a configurable parameters for AUDIO driver via KConfig?

@akumpf
Copy link
Author

akumpf commented Dec 13, 2023

The main high-level options I can imagine being helpful for basic USB audio usage are:

  • SampleRate (48000 default, but some other projects use 16000 or 8000 for speech, etc.).
  • Buffer size (affects audio latency).
  • Number of input channels (1 default, i.e. a simple microphone sending audio to the computer).
  • Number of output channels (0 default, but naturally some people will be interested in sending audio from the computer to the microcontroller).
  • Data format (standard 16 bit seems pretty universal for microcontroller audio, but there may be reasons to support 8 bit for reduced processing overhead, or 24 bit for high-end audio work).

But those are all just icing on the cake! 🍰 Getting USB Audio to work with esp-idf in any form would be fantastic.

@roma-jam
Copy link
Collaborator

roma-jam commented Dec 14, 2023

Hi @akumpf,
I have a little update.
We checked the possibility to support all the configuration, but it will take time.

Right now, the best option will be:

  1. Use the TinyUSB component directly and provide correct tusb_config.h file (please, refer this instruction: https://github.com/espressif/tinyusb/tree/release/v0.15#2-use-tinyusb-only-without-the-additions)
  2. Refer to the working example in esp-skainet: https://github.com/espressif/esp-skainet/tree/master/examples/usb_mic_recorder

It should help you to achieve the results you want.
Meanwhile, we will keep the issue open and add the feature to the plan, but the priority will be not so high.

If there is someone else, who needs the same changes, please do not hesitate to report also.

@akumpf
Copy link
Author

akumpf commented Dec 14, 2023

Thanks @roma-jam. I totally understand that audio may take some more time/testing to sort it out. Really appreciate you looking into it and providing an alternative. 👍

@akumpf
Copy link
Author

akumpf commented Jan 17, 2024

I've been continuing to experiment with getting USB audio as a simple 48khz mono/mic input. Even pulling in the example code from other projects, it never shows up for me as an audio interface on the computer. CDC, MSC, MIDI, etc. all work well, but audio does not. 🤷

I've been able to run the other Espressif_TinyUSB examples just fine (via VSCode), but when I modify those to attempt audio/UAC2 (with additional flags, descriptors, etc. as included above), it still doesn't appear to the host as an audio device.

Could the most basic USB device audio example be added, just to illustrate the minimum code/overrides needed to use ESP32 S2/S3 as an audio input?

@roma-jam
Copy link
Collaborator

Hi @akumpf,

Which OS do you use? Did you try to check the system info (Device Manager for Win or dmesg for Linux) after plugging your device? Probably there could be an answer.

Right now I would suggested:

  1. When I have made the PR for adding UAC example to esp_tinyusb: feat(usb/esp_tinyusb): Add AUDIO Device Class configuration to Konfig idf-extra-components#282 I checked it and I was able to see the device after connection to Linux host. You could refer to the changes there and try to add them localy. There is a possibility to use idf-extra-component from the local folder (with your local changes). Please refer to the Defining Dependencies in the Manifest
  2. Please, refer the example, I have mentioned before: https://github.com/espressif/esp-skainet/tree/master/examples/usb_mic_recorder. It requires specific dev board but at least it could be a good example of your goal.

Hope these points will help.
If there will be more questions, please do not hesitate to ask them.

@akumpf
Copy link
Author

akumpf commented Jan 17, 2024

Thanks @roma-jam! I was finally able to get the ESP32-S3 sending audio USB (UAC2) using the approach you suggested. It's also providing MIDI and CDC Serial at the same time, so it feels like a pretty useful combination...

I'll work with it a bit more and then hope to report back with an updated code snippet and/or pull request once it's feeling solid. 👏

@njreid
Copy link

njreid commented Apr 3, 2024

Hey @akumpf - did you get a chance to make any progress on this? Would love to try any PR you have. It seems like the combination of MIDI, UAC2, CDC serial and maybe even HID would be super useful to a lot of people.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Opened Issue is new Type: Feature Request Feature request for IDF
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants