WASAPI: Audio distortion 22kHz

[glebm]

The WASAPI audio backend introduces distortion when resampling from 22 kHZ.

SDL/src/audio/wasapi/SDL_wasapi.c

Lines 562 to 566 in 01b14e1

	if (this->spec.freq != waveformat->nSamplesPerSec) {
	streamflags |= (AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM | AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY);
	waveformat->nSamplesPerSec = this->spec.freq;
	waveformat->nAvgBytesPerSec = waveformat->nSamplesPerSec * waveformat->nChannels * (waveformat->wBitsPerSample / 8);
	}

Spectrogram recorded by @ikonomov:

Original report and more details: diasurgical/devilutionX#1390

[glebm]

Reading the docs over at https://docs.microsoft.com/en-us/windows/win32/coreaudio/audclnt-streamflags-xxx-constants, it seems that we're supposed to use IAudioClockAdjustment with AUDCLNT_STREAMFLAGS_RATEADJUST.

The AUDCLNT_STREAMFLAGS_RATEADJUST flag enables an application to get a reference to the IAudioClockAdjustment interface that is used to set the sample rate for the stream

Something like this:

    if (flags & AUDCLNT_STREAMFLAGS_RATEADJUST) {
        IAudioClockAdjustment *adjustment;
        IAudioClient_GetService(client, &IID_IAudioClockAdjustment, (void **)&adjustment);
        IAudioClockAdjustment_SetSampleRate(adjustment, rate);
        IAudioClockAdjustment_Release(adjustment);
    }

[glebm]

Looks like the issue only occurs with certain audio drivers.

One user reported this happening with the RME "Pro" Audio driver but not the default Windows one.

[sezero]

vkQuake received a report about popping / crackling audio present in 2.0.22 but not before:
Novum/vkQuake#469 - possibly because of commit 9e264b9?

[slouken]

Possibly, does backing that out fix it?

[icculus]

I couldn't hear it in the example video. 😬

I'll listen again with headphones tonight, but can someone else make sure I'm not missing the obvious crackle?

[sezero]

@icculus: is there any progress on this?

[zenxero]

@slouken
Hello, I was the one who opened the issue with vkQuake. Yes, replacing the SDL2.dll file with an older one like this fixes the audio distortion.

http://libsdl.org/release/SDL2-2.0.20-win32-x64.zip

[zenxero]

@icculus
The popping distortion sound should be the most noticeable from the axe "tings" at timestamp 2:43 in the video.

https://www.youtube.com/watch?v=oGWM9bWph2A

I'm not sure if it helps, but I've tried this on both of my Windows 10 machines (laptop and desktop), and it happens on both. They are AMD systems with Realtek audio chips. The distortion happens regardless of if I'm using the 3.5mm jack or a USB audio interface.

[icculus]

Going to look at this today; my suspicion is that it isn't our resampler in general but likely that our resampler dropped precision to float from double, but I could be wrong.

[sezero]

but likely that our resampler dropped precision to float from double, but I could be wrong.

IIRC, did that not happen in git master? The issue is reported for 2.0.22.

[icculus]

IIRC, did that not happen in git master? The issue is reported for 2.0.22.

Oh, hmm, yes.

[icculus]

I went back and checked again...I have to be honest, I can't hear it in the youtube video even with headphones on, I can't hear it in a Linux build using the latest git of vkQuake and SDL. It sounds the same to me in both versions in the video.

My hearing is not great, but I don't hear any crackle in it at all.

Here, this is the audio output of the youtube video during the axe strikes. Can you tell me which one is from 2.0.20 and which is from 2.0.22?

https://www.dropbox.com/s/gvkbrepl3vmydd8/axes.wav?dl=0

[icculus]

(I don't see anything that looks significantly different between the waveforms in that .wav file, when loaded in Audacity, either, but looks can be deceiving in this regard.)

[vladkosi]

They look slightly different (right side is 2.0.22), but I can't hear the crackling.

[zenxero]

The one on the right has the crackling. I can hear it in the audio file played through audacity.

[zenxero]

These large spikes here are the popping/crackling sound

[vladkosi]

I can reproduce it at 48000 Hz. Left side is 2.0.20, right side 2.0.22:

[icculus]

What do you do to make Audacity show that view? I just get a simple waveform in mine.

[vladkosi]

https://manual.audacityteam.org/man/spectrogram_view.html

[vladkosi]

Here is the recording: 1.zip

[vladkosi]

Seems like the 48 kHz output is the most broken. 96 kHz and 192 kHz don't look as bad:

[icculus]

I dunno, I can see it in all resamples. Here it is, pushed through SDL_sound's playsound app, which lets you resample before feeding to SDL, then I used SDL's disk audio driver to write PCM to disk instead of to an audio device...

(Original wav at 44100, playback with no resampling, then 48000, 96000, 192000)

[icculus]

...although it definitely elongates in lower sampling rates if you zoom in to look at it...

[icculus]

I really can't believe this small change in the waveform is making this much of a difference.

[vladkosi]

You can switch to dB view to see the change more clearly: https://manual.audacityteam.org/man/audacity_waveform.html#db

[icculus]

Just for comparison, SDL's built-in resampler vs SDL using libsamplerate...

...this has got to be a filter issue...

[icculus]

Okay, I literally dropped in the filter generation code from resample-1.8.1 and it still happens, so the silver lining here is knowing that less of my original code is untrustworthy now. :)

Poking the resampling implementation now that I can assume the filter data is probably good.

[icculus]

I still haven't figured this one out; I'm fried, I'll try again in the morning.

[icculus]

I've been looking at this forever and can't see what's wrong; I'm spending too much time on this, so I'm going to come back to this later in the milestone.

[icculus]

Wondering if these old bug reports are related: #2715 #2716

[ericoporto]

Sorry to ping, I can't reproduce this but am curious if there's any updates on this.

[icculus]

Went back and tested this with SDL3, and we have fixed this.

I don't have the original data from the first time I generated these graphs, but here's a resampling from the first half of the axes.wav I posted up above:

This is what it looks like with SDL2's resampler...the garbage is less severe, either because we made fixes or this is working from data we already resampled once, I don't know, but you can see there's still garbage in the spectrogram in any case, especially at 48000Hz...

So I think we're good to go in SDL3.

There's no chance we're backporting this to SDL2, though, as all this code was completely rewritten. Even discounting the idea of migrating bugfixes to SDL2 here, I don't think I could even drop the new resampler code into SDL2 as a complete replacement, in any reasonable way. But for games not migrating to SDL3, sdl2-compat will take advantage of the new resampler.