Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDL3 audio conversion redesign #7571

Merged
merged 1 commit into from
Apr 27, 2023

Conversation

icculus
Copy link
Collaborator

@icculus icculus commented Apr 2, 2023

(This is just about ready to merge, but it's a gigantic change and I need to, at a minimum, get the buildbots happy with it again, so this is still a draft.)

This is the rewrite of the audio converters, the first big step towards what I envision for SDL3's audio subsystem.

SDL_AudioCVT is gone, even internally.

There is an internal function called ConvertAudio(), which does all the work of SDL_AudioCVT except resampling. ResampleAudio() does that. The two are combined in SDL_AudioStream.

libsamplerate is gone.

I suspect our resampler is finally Good Enough, knock on wood. Most people never used the libsamplerate support; even if it was compiled in and available at runtime, you had to force it with an SDL hint to enable it. This was always meant to be an in-case-of-emergency fallback, and it complicated the conversion code to have an abstract interface to support the option, so I yanked it. If we need to, we can put it back in, but I'm hoping we don't need to.

Cleanups and improvements to audio conversion interfaces.

Just a lot of simplification and tweaks across the board.

SDL_AudioStream can now change its output format/rate/channels on the fly!

I buried the lede, this part is awesome. I need this for the Big Plans for SDL3 audio, but even in normal app use, it's fun to change an audiostream's sample rate on the fly, like this program does:

sdl3-dynamic-audiostream.mp4

Resamples in chunks work correctly now.

Issue #7358 is gone, the redesign fixed it; feed the same data in blocks or as one big blob and the final output will have the same checksum.

Still TODO:

I added a ton of FIXMEs (both to this work and other things I saw as I moved around in this code). Notably, AudioStreams allocate a much larger work buffer than they probably need, just for laziness at the moment. We can calculate that more carefully.

Valgrind saw an off-by-one issue I need to track down, too.

Related issues

Fixes #7378
Fixes #7358

Reference Issue #6632

(Maybe others, too.)

@icculus
Copy link
Collaborator Author

icculus commented Apr 3, 2023

Realized that if someone wanted to build an SDL_mixer-like thing on top of SDL_AudioStream, they're probably going to want to change the input format, too, instead of just the output format...so they can reuse streams with different audio files over time, but also because my original trick in that video was to change the output sample rate, but this will not be under application control when we start attaching streams directly to devices, so they'll want to use the input end to change the frequency for pitch effects and speed.

So SDL_SetAudioStreamOutputFormat is gone, and SDL_SetAudioStreamFormat() replaces it, letting both ends change.

If changing the data format or channel count, it'll convert its history/future buffers so that resampling continues to produce correct audio. If the input sample rate changes...it doesn't do anything. The next read will drain/fill the future buffer as necessary to produce right-side padding for the resampler, and we just survive with the history buffer we've got in place at the wrong frequency, and assume any distortion will be brief and imperceptible, since the pitch and speed is changing at the same time, and the history buffer will correct itself quickly as data shuffles through the queue.

This is all to say: I don't think it will be a big deal, but we can always choose to take the hit to resample the history buffer if we decide it's necessary later.

@icculus icculus force-pushed the sdl3-audio-conversion-redesign branch 2 times, most recently from 85efbe0 to a42d548 Compare April 25, 2023 03:02
@icculus icculus marked this pull request as ready for review April 27, 2023 20:06
@icculus icculus force-pushed the sdl3-audio-conversion-redesign branch from 91f1060 to 427258d Compare April 27, 2023 20:08
@icculus
Copy link
Collaborator Author

icculus commented Apr 27, 2023

Ok, I fixed the last issue, squashed this down into a single commit, and this is ready to go, once the build checks finish.

@icculus icculus force-pushed the sdl3-audio-conversion-redesign branch from 427258d to 80c2c6f Compare April 27, 2023 21:00
- SDL_AudioCVT is gone, even internally.
- libsamplerate is gone (I suspect our resampler is finally Good Enough).
- Cleanups and improvements to audio conversion interfaces.
- SDL_AudioStream can change its input/output format/rate/channels on the fly!
@icculus icculus force-pushed the sdl3-audio-conversion-redesign branch from 80c2c6f to 045535e Compare April 27, 2023 21:14
@icculus
Copy link
Collaborator Author

icculus commented Apr 27, 2023

Here it goes! holds breath, pushes button

@icculus icculus merged commit e5a6c24 into libsdl-org:main Apr 27, 2023
@icculus icculus deleted the sdl3-audio-conversion-redesign branch April 27, 2023 22:35
@icculus icculus restored the sdl3-audio-conversion-redesign branch August 10, 2023 03:02
@icculus icculus deleted the sdl3-audio-conversion-redesign branch July 2, 2024 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant