Expose AudioStream + AudioStreamPlayback methods to GDExtension

[stechyo]

Describe the project you are working on

I would like to integrate Steam Audio with Godot, and GDExtension seems like the best way to go about it.

Describe the problem or limitation you are having in your project

The methods that Godot exposes to GDExtension are virtual versions, which are fine for making completely new audio streams but not for e.g. passing the audio from an existing stream through custom processing. See the example below (and if I'm doing something wrong, let me know):

// stream_playback is a Ref<AudioStreamPlaybackMP3> in this case
AudioFrame input[frames];
int mixed = stream_playback->_mix(input, rate_scale, frames);
// mixed == 0, all audio frames in `input` are zero

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Calling _mix() doesn't do anything, what I think I want to do is to call mix() (no underscore!), get the resulting audio frame array, and pass it through SteamAudio processing.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

We could expose mix() and perhaps other methods. The way I understand this is done is by referencing them in the _bind_methods() func in audio_stream.cpp. Exposing mix this way, however, exposes it to GDScript/C# (if there's another way to expose mix to just GDExt let me know), and this is something which the maintainers may not want to do (esp. seeing as we'd have to make AudioFrame not just a simple C++ struct but something that's more compatible with GDScript.)

If this enhancement will not be used often, can it be worked around with a few lines of script?

I don't believe so, but please let me know if there's another way to go about this.

Is there a reason why this should be core and not an add-on in the asset library?

It must be in the core code afaik.

[AThousandShips]

See also:

• Expose more audio methods to be able to generate waveform previews #127

[darkcanopy]

The fact that Steam Audio is now open source means that it would be really nice to be able to easily use official Godot with a Godot Steam Audio gdextension (like stechyo here has created.)

[stechyo]

Closing this as the related PR got merged.