TinyUSB Audio issue with rp2040

[TheMindVirus]

Operating System

Windows 10

Board

Raspberry Pi Pico RP2040 and Adafruit Macropad RP2040

Firmware

Pico C SDK Firmware Examples (and the same in Arduino IDE)

What happened ?

USB Audio over TinyUSB Audio in the Pico-SDK produces distorted bit-crushed noise over any audio data passed through it or generated. Identical behaviour is observed on different RP2040 boards so the issue looks like it's "i2s_dummy_buffer" or something else causing unwanted noise on all USB Audio Endpoints. Before you point back to them, I have already raised this issue with Pico-SDK maintainers and they pointed me here instead of maintaining their fork.

Set up
Standard Raspberry Pi Pico RP2040 and also Adafruit Macropad RP2040
Windows 10 Pro with Pico SDK, Recommended ARM GCC and additionally Arduino IDE
Windows 10 Pro Runtime OS Device Manager and additionally Windows 10 on ARM with the same universal device behaviour

Describe the bug
Please see: raspberrypi/pico-sdk#653 for more information
"i2s_dummy_buffer" @ https://github.com/raspberrypi/tinyusb/blob/d49938d0f5052bce70e55c652b657c0a6a7e84fe/examples/device/audio_4_channel_mic/src/main.c#L394
https://github.com/raspberrypi/tinyusb/blob/d49938d0f5052bce70e55c652b657c0a6a7e84fe/examples/device/audio_4_channel_mic/src/main.c#L419
Appears to be causing bit-crushing distortion to any wave applied through it, especially 2 in 2 out USB audio interfacing

How to reproduce ?

1. Download the Pico SDK and Build the usb examples (especially audio_4_channel_mic from tinyusb and other relevant audio tests)
2. Ideally adapt the example to create "audio_2i2o", a 2 input 2 output Audio Unit USB interface with the Pico or RP2040 board
3. Deploy the example UF2 file (converted from .elf with ELF2UF2) onto the Pico, look in Device Manager and Record it in FL Studio. Optionally Audacity can be used for the recording. Increase the volume of the sample to 300% to make the interference pronounced.

Debug Log

No log available from the Pico while it is processing audio over USB, other than all TU_LOG2() statements included in the audio_4_channel_mic example (code reaches all the segments it needs to). Same issue with logging from Macropad but identical data can be observed in the recorded audio sample (which could eventually be used as a trace).

To be clear, this is only an issue on your repository because the pico-sdk and raspberry pi tinyusb maintainers aren't maintainers.
They told me to open the same issue here. I see UVC 1.5 is upcoming, great work on it so far.
I hope the data I captured is useful to you for fixing USB Audio.

Screenshots

The signal shown in the screenshot below is meant to be a clean Sine Wave and is instead Frequency Modulated chaos:

The samples included in the following zip file were a second attempt at testing USB Audio and similar distortion can be heard
or observed using an audio editor:
PicoSample.zip

[TheMindVirus]

Here is the current status of uac2_headset on the latest pico-sdk with the latest tinyusb:

I play a low frequency sine wave through it just so you can see the digital artifacts that seem to be occurring at random intervals.
It is almost as if extraneous data is being written to the same USB endpoint, or it needs to be buffered again to reduce jitter.

On another note I built the new video_capture example and that seemed to work well on WoR on Pi4:

tinyusb_dev_video_capture_uf2.zip

[hathach]

Thanks for opening the issue, UAC2 is tricky, and there is still lots of work to get it working flawlessly. To make the issue reproducible for me and others, can you

• Provide the compile-ready example based on one of the audio example. adapt the example to create "audio_2i2o" can have lots of variants. While we want to make sure to test with the same firmware on the same hardware as much as possible. The code should be reviewable so that other can comment on specific line of code as well rather than an zip file. The easiest way is
  • Fork this repo, checkout a new branch
  • Add your-own-example based on stock one
  • Push and post it here.
• For testing: please specify which application you use, and exact steps to capture and the audio sample. I use Linux and not familiar with any audio processing app, if you know any linux app that could do the same, it will help to speed up my testing/fixing cycle.

PS: Thanks for testing out the UVC, though we should focus on the UAC2 in this issue, UVC has its own checklist issue

• Video UVC 1.5 check list  #1142

[TheMindVirus]

I have made the following fork of TinyUSB combined with Pico-SDK: https://github.com/TheMindVirus/tinyusb
It's actually Pico-SDK but the fork is TinyUSB for tracking it here. It is a snapshot, no Pull Request is to come from it.
It is also not an archival fork/branch but it should almost be treated as such. I have versions of Pico-SDK and TinyUSB
which are completely different but the version numbers says they're identical. (Cached vs. Non-Cached Version Hashing?)

GitHub was a nightmare about removing submodules but also having local folders with the same name.
I removed all submodules, moved the actual files to a temporary staging area, deleted the in-branch folders and
added them back in again with core.autocrlf = false. All of these things might mean that the fork is not an exactly
identical copy of my checkpoint#3 that I have locally. I have built and tested the new clone somewhere else on my machine
and it seems to build ok, but not sure about how it will work on a different machine (especially one which isn't x86->arm32).

At one point when deleting the branches, GitHub warned me that something upstream will be deleted when this is
probably not what it should have been doing or asking to do. Please tell me if there is something missing from the
upstream branches. As a precaution, I have since removed the link to the upstream repo which .git, .github, submodule .git, submodule .github, GitHub for Windows and github.com all disagreed on.

The TinyUSB included more closely matches Pico-SDK's smaller version.

As for testing, I have included information about what I used for my testing but I haven't included exact steps as these may
vary between tests and also in practice. A general usability test will pass if you are able to just plug in the device, flash the .elf/.uf2 to it and immediately start using it in the Host OS's native applications with no additional drivers required.
A similar thing can be said for an Internet of Things device using a Cloud-based firmware download platform.

Anyway, I need a break now. Please let me know if there are updates to Audio/Video for RP2040 at a later stage.
Hope this helps. A.

[hathach]

I still don't get how to reproduce your issue

1. Which example firmware I should compile, please provide link to your fork example
2. Why do you need to fork the pico-sdk, did you make any changes to the sdk.
3. Can you provide audio tool and steps to reproducing the capture

Note: all the info should be posted directly here.

Please let me know if there are updates to Audio/Video for RP2040 at a later stage.

just follow the development on this repo or checkout the release note.

[TheMindVirus]

The issue mainly seems to relate to Audio Formats in the USB Descriptor and what's expected by apps.
ASIO issues have occurred but the jitter has occurred with non-ASIO apps and ASIO has sometimes worked.
It may have something to do with the mic interleaving 1 channel from 2 but that's usually a normal case.
It may also have something to do with extra buffering when it does work, but will also temperamentally
throw device exclusivity and sharing errors (not very conducive to the audio workflow but I see why).

Notes.txt:

* File Open Errors with ASIO and other applications such as Audacity
* Audacity momentarily worked nicely, so did FL Studio's built in Recorder but not Edison VST or Voice Recorder
* OBS Got 1ch Audio In but might have been trying to interleave channels???
* Suggest Audio_2i2o? The second channel is often phase or amplitude inverted to change mono to stereo

PicoTinyUSBAudioTest.zip

In response to your questions:

1. The tinyusb_dev_uac2_headset.uf2 firmware I used for testing is included in the .zip file attached. It is also on the fork.
2. The pico-sdk is the building framework I'm using to also build TinyUSB. I added pico-toolchain and added batch files
   (and in Checkpoint 2 I have more heavily rebased the SDK and TinyUSB but these changes are not included in Checkpoint 3).
3. If you still want steps to reproduce it, I used FL Studio ASIO and ASIO4ALL control panel so I could select which device I wanted to play back to and record. These are also in Windows Media Player settings and Sound Control panel for Windows (and Linux applications) but are more cumbersome to find and use. (e.g. 2x Audacity doesn't let you play and record simultaneously). I even tried Windows Media Player going to OBS Studio and some jitter occurred. Audacity and FL Studio did at one point work but were slightly picky when it came to sharing the preview USB Audio Device. The jitter is still visible in the recorded waveform at source though so something is happening with invalid format.

[AaronD-GH]

Since this is still open, with seemingly insufficient documentation, here's what I posted to the RPi forum a couple days ago:
https://forums.raspberrypi.com/viewtopic.php?t=341735
My entire project is over there, after paring it back down to what I think is equivalent to the uac2_headset example. (that's what I get for assuming that the examples "just work")

---

Essentially, it works immediately after plugging it in. It enumerates just fine, it appears on my Pi4 (Raspberry Pi OS, fully updated) as an audio device, and an oscilloscope attached to a GPIO toggle for every USB audio frame shows the correct frequency (1 edge per ms) until (presumably) the host audio driver stops sending silence.

The first app to play audio, works too. The 'scope wiggle restarts, and the loopback shows what looks like a copy of what was played. (didn't actually check it in detail though)

BUT! Once that first app stops playing (even pause is enough), every subsequent attempt to use that audio device throws errors, regardless of which app it comes from, and the 'scope wiggle never restarts. The only way I've found to recover is to unplug and replug (or blink the RUN pin), let it re-enumerate, which always succeeds, and then I'm back to the one opportunity to play something until it bombs out again.

More details are in the thread linked above.