STM32F4 Asynchronous USB Audio Firmware

This project is based on STM32469I-Discovery "AUDIO_Standalone" Example, and I largely rewrite the USB audio class library provided by ST.

• Folder structure, startup script, linker file are generated by setting up a dummy project (USB, SAI, DMA, I2C, GPIO enabled) with STM32CubeMX.
• Drivers/BSP/ and Middlewares/ST/STM32_Audio/ are copied from STM32CubeF4 Repositiry.
• Files generated by STM32CubeMX in Inc/ and Src/ are replaced with those from STM32469I-Discovery "AUDIO_Standalone" Example.

Table of Contents

• Feature
• For User
• For Developer
• Firmware Architecture
• Technical Highlight
• Background Knowledge

Feature

• Using asynchronous mode for isochronous transfer.

  • 1 ISO OUT endpoint for PCM data, 1 ISO IN endpoint for feedback.

• Support 16-bit / 24-bit, 44.1 kHz / 48 kHz / 96 kHz, stereo PCM audio.

• Support mute, volume, frequency control from USB host.

  • Mute and volume control commands are passed to CS43L22 codec.
  • Frequency control commands change the PLL settings on MCU to generate different MCLK for SAI block.

• Implement USB Audio Class 1.0 on USB OTG Full-speed core.

• LED status indicator

  Green LED : ON when playing.

  Orange LED : ON when buffer overrun (with audible distortion).

  Red and Blue LED : Depend on audio data frequency.

  Frequency	Red LED	Blue LED
  44.1 kHz	ON	-
  48 kHz	-	ON
  96 kHz	ON	ON

For User

Install basic tools

# Arch Linux
pacman -S make arm-none-eabi-gcc

# Ubuntu
apt-get install make gcc-arm-none-eabi

Compile the code

make

Flash the binary

Method 1 : Use open-source version of stlink

# Install stlink you don't have it
# Arch Linux
pacman -S stlink
# Ubuntu
apt-get install stlink-tools

# Connect your board from the ST-LINK connector to PC. Then, run
make flash

Method 2 : Use STM32CubeProg

1. Connect your board to PC from the ST-LINK connector, open STM32CubeProg, click "Connect".
2. Go to "Erasing & Programming" tab, click "Browse", choose the binary build/f469-usbaudio-ex2.bin.
3. Click "Start Programming".

Use the USB audio device

1. Connect your board to PC from the MicroUSB connector (CN13). (Not the ST-LINK MiniUSB connector).

   The ST-LINK MiniUSB connector should still be connected to PC because the board is powered from this port.

2. There will be something like "USB Audio Speaker" appears on PC. Set it to default audio device and play music with that device.

   ⚠️ Warning

   Set volume to the lowest level before plug-in a headphone. The firmware is not well-tested to guarantee safe initial volume on all platforms.

• On Linux, pactl list short sinks recognizes the USB audio device as the following :

  alsa_output.usb-STMicroelectronics_STM32_AUDIO_Streaming_in_FS_Mode_<serial_number>-00.analog-stereo
  

For Developer

Development tools

• make The build system.
• arm-none-eabi-gcc, arm-none-eabi-gdb, arm-none-eabi-newlib ARM cross-compiling and debugging toolchain.
• stlink For firmware flashing and debugging.
• openocd Debugging tool.

# Arch Linux
pacman -S make arm-none-eabi-gcc arm-none-eabi-gdb arm-none-eabi-newlib stlink openocd

If you use VSCode, the following extensions may help (not necessarily needed) :

• C/C++ by Microsoft
• Cortex-Debug by marus25

Compile the code

make

To clean up build/ directory,

make clean

Flash the binary

make flash

This is the shorthand for :

st-flash --reset write $(BUILD_DIR)/$(TARGET).bin 0x08000000

Debug the code

I debug on VSCode with Cortex-Debug extension.

First, connect your board to PC from the ST-LINK connector.

On Debug tab in VSCode, choose Debug (OpenOCD) config and start. Then, you'll see the familiar debugger running.

• SVD file used to display peripheral registers is taken from https://github.com/posborne/cmsis-svd/blob/master/data/STMicro/STM32F469.svd

Debug USB

I use wireshark to inspect USB packets.

# Arch Linux
pacman -S wireshark-qt

To enable USB sniffing, one needs to load usbmon kernel module. It's built in Linux kernel.

sudo modprobe usbmon

Then, run wireshark with root so that it can intercept USB packets.

sudo wireshark

Some useful filters :

# Device address
usb.device_address == 123
  
# Endpoint 1, IN direction
usb.endpoint_address == 0x81
  
# Frame length. Valid feedback packet is 83 bytes (80 bytes header and 3 bytes data).
frame.len == 83
  
# bRequest can be used to filter control packets
usb.setup.bRequest == 11

Useful commands

• List USB devices

$ lsusb
Bus 002 Device 067: ID 0483:5730 STMicroelectronics Audio Speaker

• USB device details

$ lsusb -D /dev/bus/usb/002/067
Device: ID 0483:5730 STMicroelectronics Audio Speaker
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x0483 STMicroelectronics
  idProduct          0x5730 Audio Speaker
  bcdDevice            2.00
  iManufacturer           1 
  iProduct                2 
  iSerial                 3 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x007c
    bNumInterfaces          2
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
...

• Monitor audio playback status (change the X in cardX to the number of actual device)

$ watch -n 1 cat /proc/asound/cardX/stream0                                                    
Dragonode Audio Venus DAC at usb-0000:00:02.0-3, full speed : USB Audio

Playback:
  Status: Running
    Interface = 1
    Altset = 2
    Packet Size = 432
    Momentary freq = 47569 Hz (0x2f.918c)
    Feedback Format = 10.14
  Interface 1
    Altset 1
    Format: S16_LE
    Channels: 2
    Endpoint: 1 OUT (ASYNC)
    Rates: 44100, 48000, 96000
  Interface 1
    Altset 2
    Format: S24_3LE
    Channels: 2
    Endpoint: 1 OUT (ASYNC)
    Rates: 44100, 48000, 96000

• Kernel messages

$ dmesg
[72898.617745] usb 2-1: new full-speed USB device number 67 using xhci_hcd
[72898.759182] usb 2-1: New USB device found, idVendor=0483, idProduct=5730, bcdDevice= 2.00
[72898.759188] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[72898.759190] usb 2-1: Product: Venus DAC
[72898.759193] usb 2-1: Manufacturer: Dragonode Audio

Firmware Architecture

• Incoming (USB) to outgoing (I2S) buffer chain (bit width, byte alignment, config) TBD
  • USB buffer : 16-bit or 24-bit frame, little-endian -> Audio buffer : 32-bit frame, right-aligned, little-endian -> 32-bit DMA -> SAI FIFO (32-bit width)
• Feedback (method & algorithm) TBD
  • Calculate by timer.
  • Calculate by remaining buffer size.
• USB Interrupts & data flow TBD
• Relationship between USB class driver, main program, audio codec and SAI. TBD

Technical Highlight

• PCD_HandleTypeDef->Init->Sof_enable (Src/usbd_conf.c:274) must be 1 to enable SOF (Start-of-frame) interrupt.

• The second parameter of HAL_PCDEx_SetTxFiFo(&hpcd, 1, 0x60) (Src/usbd_conf.c:287) must be > 0 so that there is FIFO to store Tx data (in this case, the feedback data). If it's 0, there will be bugs described here.

• Feedback data byte order. TBD

• When to recv / send data ? USB IN / OUT / SOF token. TBD

• USB volume to Codec volume mapping. TBD

• Extending 16-bit to 24-bit

  • DMA

    Set DMA_HandleTypeDef.Init.PeriphDataAlignment to DMA_PDATAALIGN_WORD and DMA_HandleTypeDef.Init.MemDataAlignment to DMA_MDATAALIGN_WORD.

    This means we need to wrap 24-bit audio sample in 32-bit structure.

    A DMA transaction consists of a sequence of a given number of data transfers. The number of data items to be transferred and their width (8-bit, 16-bit or 32-bit) are software-programmable.

    RM0386 Reference Manual - 9.3.3 DMA transactions

  • SAI

    Set SAI_HandleTypeDef.Init.DataSize to SAI_DATASIZE_24.

    Set SAI_HandleTypeDef.FrameInit.FrameLength to 128.

    Set SAI_HandleTypeDef.FrameInit.ActiveFrameLength to 64.

  • USB

    • Set USBD_MAX_NUM_INTERFACES(Inc/usbd_conf.h:58) to 2.

    • Add an Audio Streaming Interface Descriptor with bAlternateSetting set to 2.

      Set bSubFrameSize to 0x03 and bBitResolution to 0x18.

    • Open OUT EP with max packet size of 24-bit / 96 kHz

      USBD_LL_OpenEP(pdev, AUDIO_OUT_EP, USBD_EP_TYPE_ISOC, AUDIO_OUT_PACKET_24B);

    • Handle SET_INTERFACE request in Setup stage. e.g. Set a flag to let other processes know it's 24-bit data.

    • Extend 16-bit or 24-bit data to 32-bit

      Note that ARM is little-endian. Consider the following :

      uint8_t tmpbuf[2] = { 0x34, 0x12 };
      // *(uint16_t*)&tmpbuf[0] is 0x1234

    • USB Device Rx FIFO size must be sufficiently large ( > Max audio payload size + USB Header ). At the same time Tx FIFO size may need to be shrunk so that total FIFO size doesn't exceed the limit (In Full-speed : 1.25 Kbytes, 0x140 words). Ref : STM32 Cube USB Host wmaxpacketsize problem

      HAL_PCDEx_SetRxFiFo(&hpcd, 0x110);
      HAL_PCDEx_SetTxFiFo(&hpcd, 1, 0x10);

Background Knowledge

USB & USB Audio Class

• Асинхронное USB аудио на STM32 I follow most of the steps in this forum post except for using timer to calculate feedback values.
• borgestrand/sdr-widget An USB audio class implementation for Atmel AT32UC3A3. They use remaining buffer size to calculate feedback data, which is easier and more portable since it doesn't require specific hardware feature like the timer solution.
• USB Specification v1.1
  • 5.10.4.2 Feedback - Describe the 10.14 format and explain the theory to calculate feedback data using a timer. (So, the timer solution is actually the standard answer.)
  • Chapter 8 Protocol Layer - Get concepts about USB packet and transaction.
  • USB Made Simple has a simplified version that is easier to read.
• USB Device Class Definition for Audio Devices v1.0 For writing audio class descriptors and class-specific control logic.
• 成大資工 Wiki - USB USB 基礎介紹（中文）
• USB Audio 簡介 (UAC 1.0) （中文）
• USB 之一 USB2.0 规范详解 第一部分 簡化版的 USB Specification v2.0（中文）
• UAC2 Feedback Mechanism

STM32

• UM1021 STM32 USB OTG Library User Manual The architecture of ST's USB library and how to use it.

• STM32 之三 标准外设版USB驱动库详解 大致與 UM1021 的內容相同（中文）

• UM1725 Description of STM32F4 HAL and LL drivers Chapter 47 and 48 are USB-related. The UM1201 USB library relies on these drivers to do low level work.

• RM0386 STM32F469xx and STM32F479xx Reference Manual The documentation for the MCU itself, it's mostly about peripheral registers. HAL and LL drivers translate software commands to electrical actions by manipulating these registers.

  Important chapters :

  • 9 Direct memory access controller (DMA)
  • 32 Serial audio interface (SAI)
  • 35 USB on-the-go full-speed/high-speed (OTG_FS/OTG_HS)

• ChibiOS forum - Usage of USB driver in isochronous mode (STM32) Register level details about gotchas in implementing audio class with STM32 USB hardware stack. After some investigations, I think the problems described there seem to be solved in the latest STM32F4xx LL / HAL driver.