Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SDL3 Audio backend #6002

Draft
wants to merge 132 commits into
base: master
Choose a base branch
from
Draft

Implement SDL3 Audio backend #6002

wants to merge 132 commits into from

Conversation

hwsmm
Copy link
Contributor

@hwsmm hwsmm commented Sep 23, 2023

Please note that this is my first proper C# project. I got interested by #2784, and I started learning C# just because I wanted to try making it.
I ended up making it work somehow, but I just have no idea about what I need to do to get further.

Description

This PR separates AudioManager into BassAudioManager and AudioManager, and adds SDL3AudioManager and some more components.
Changing AudioDriver to SDL3 in framework.ini enables this feature. BASS is used as default without that option.

Track/Sample instances now actually have their raw audio data, so they are managed by C# rather than by BASS. Due to this, it's technically SDL audio implementation, but SDL doesn't manage samples/tracks. Most logics including mixing are now in C# code, so it should be fairly easy to change to another audio library as long as it supports queueing audio directly like SDL.

This mainly benefits Linux users as SDL supports PipeWire, PulseAudio, ALSA and JACK natively, whereas BASS only supports ALSA.

Choosing audio decoder

Either BASS or FFmpeg is usable. If you want to use FFmpeg, you need to bring proper FFmpeg binaries. osu!framework uses cut-down FFmpeg only for videos. libswresample is also needed.

You can enable FFmpeg by removing BASS init lines in SDL3AudioDecoderManager. BASS is used as default otherwise.

Used libraries

SoundTouch.NET is used to adjust track tempo. NAudio is used to apply BiQuad filters, adjust frequency of tracks and samples, and perform FFT for Waveform/CurrentAmplitude. SDL is used to push audio to actual audio server. (No native libraries added!)

They are all open-source, so it should be much easier to track down bugs.

Tested platforms

Tested on Windows 11, Linux and Android. Should be also usable in iOS and macOS, but I don't have any Apple device to test on.

Notable changes to existing components

  • VideoDecoder can now decode audio (meant to separate, but it would make too much diff)
  • Waveform is not dependent on BASS anymore, but performance in benchmarks is about 1.3x worse (70ms versus 53ms on previous implementation)

TODO

This PR requires no change in osu! itself, but may require in future as the game uses ManagedBass.Fx directly.

What's not working

  • Any audio effects other than BiQuad ones

Some quirks

  • Audio glitches a lot when GC is happening and buffer is small

Needs to be done in future

  • Change to the original SoundTouch instead of .NET port to get some optimization: my old code used this, and it works as good as .NET one, but I didn't want to add a native library in this PR.
  • Make VideoDecoder abstract, and separate audio part from it
  • BASS tests are reused to reduce diffs, may need to separate in future.

@smoogipoo
Copy link
Contributor

This is super cool, though I don't see osu! using this in the near future if ever. I'd see this implemented as an additive nuget package, and exposed through some way that isn't the framework config. For example, it could be included in HostOptions or simply as a virtual method in Game.

@peppy
Copy link
Sponsor Member

peppy commented Sep 25, 2023

I'd actually like to experiment with this and see how good support is for the upcoming changes I want to make with WASAPI initialisation. So I don't want to throw this out. Moving away from bass would be a huge consideration, but I wouldn't throw it away.

I'm going to mark this as a draft as I don't see it getting reviewed or merged anytime soon, but I still useful to have around as a reference for what is involved in making this work, and potential performance / latency cross-checking in a future.

@peppy peppy marked this pull request as draft September 25, 2023 04:05
@hwsmm
Copy link
Contributor Author

hwsmm commented Sep 29, 2023

I'll maintain this until some of you have anything to do with this, mostly because I am now so used to playing the game with these patches, I can't really go back to BASS...
FWIW, on Linux with Pipewire at very small buffer, hitsound latency was almost on par with patched wine which osu! linux players mostly use. It produces some artifacts, so shouldn't be default, though.

@hwsmm hwsmm force-pushed the sdl-audio branch 2 times, most recently from 8c2a3e9 to 43d8469 Compare August 6, 2024 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants