Implement audio stream playback parameters. #86473

reduz · 2023-12-23T16:41:39Z

Implements a way for audio stream playback to be configured via parameters directly in the edited AudioStreamPlayer[2D/3D].

Currently, configuring the playback stream is not possible (or is sometimes hacky as the user has to obtain the currently played stream, which is not always immediately available).

All you can do is configure the AudioStream itself, which affects all instances in the game. With this PR, configuring playback parameters per each stream player is possible, giving users far more flexibility and freedom to configure playback or animate parameters.

This PR only implements this new feature to control looping in stream playback instances (a commonly requested feature, which was lost in the transition from Godot 2 to Godot 3). But the idea is that it can do a lot more:

If effects are bundled to the stream, control per playback instance parameters such as cutoff or resoance, or any other exposed effect parameter per playback instance.
For the upcoming interactive music PR (Add interactive music support #64488), this exposes an easy way to change the active clip, which was not possible before (so this PR is a prerequisite for that one).
For the upcoming parametrizable audio support (Add procedural and parametrizable audio support godot-proposals#3394) this allows editing and animating audio graph parameters.

Screenshot of how it looks:

Supersedes #65797, implements godotengine/godot-proposals#3120.

modules/vorbis/audio_stream_ogg_vorbis.h

modules/minimp3/audio_stream_mp3.h

modules/vorbis/audio_stream_ogg_vorbis.h

KoBeWi · 2023-12-23T17:03:55Z

Supersedes #65797 I guess.

doc/classes/AudioStream.xml

reduz · 2023-12-23T18:06:31Z

@AThousandShips Thanks!! Merged and squashed.

reduz · 2024-01-08T14:22:38Z

Did some changes, looping is now something you can check to override, to make it less confusing.

I think this is ready for review and merge.

aaronfranke · 2024-01-09T02:49:47Z

I actually implemented per-player audio looping in The Mirror's fork of Godot a few months ago so that audio players can be looped by users, but I didn't know if this was desired to have upstream. So from that perspective, it's great to have this in the engine, less stuff for us to maintain :P

From the GLTF perspective, the latest iteration of the audio standard has looping as a part of the audio source (what Godot would call the audio stream), so this PR isn't useful for GLTF audio, but it doesn't hurt it either.

reduz · 2024-01-09T13:07:19Z

@aaronfranke The problem we have in the audio side is that parameters exposed at stream level are very tricky because:

Streams many times are imported, so you can't edit them directly.
Even if you can edit them directly, they are shared across all players.

So, this makes editing or interacting with local instances very hard. This PR is created to make this easy and allow exposing parameters that can be edited locally. Additionally, as they appear at node level (similar to how animationtree works), you can animate them and edit them more freely.

servers/audio/audio_stream.h

doc/classes/AudioStream.xml

KoBeWi · 2024-01-09T20:03:00Z

When you set a parameter in AudioStreamPlayer and then uncheck it, it will be stored in the scene as null.

modules/minimp3/audio_stream_mp3.cpp

servers/audio/audio_stream.cpp

scene/2d/audio_stream_player_2d.cpp

modules/minimp3/audio_stream_mp3.cpp

modules/minimp3/audio_stream_mp3.h

modules/vorbis/audio_stream_ogg_vorbis.cpp

modules/minimp3/audio_stream_mp3.cpp

scene/2d/audio_stream_player_2d.cpp

KoBeWi · 2024-01-10T12:23:18Z

Now the parameter is always null and can't be changed.

adamscott · 2024-01-10T14:08:59Z

doc/classes/AudioStream.xml

@@ -28,6 +28,12 @@
 			<description>
 			</description>
 		</method>
+		<method name="_get_parameter_list" qualifiers="virtual const">
+			<return type="Array" />


Suggested change

<return type="Array" />

<return type="Dictionary[]" />

Based on the failing CI prompt.

Might make sense to also slightly tweak the description below after this change since "array contains dictionaries" will be redundant.

ellenhp

LGTM for audio. I have not built or tested so it would be good if someone else could confirm that they've played with it, but the approach seems sound and I think this'll make a lot of people really happy.

ellenhp · 2024-01-10T21:27:35Z

doc/classes/AudioStream.xml

@@ -28,6 +28,12 @@
 			<description>
 			</description>
 		</method>
+		<method name="_get_parameter_list" qualifiers="virtual const">
+			<return type="Array" />


Might make sense to also slightly tweak the description below after this change since "array contains dictionaries" will be redundant.

reduz · 2024-01-11T09:15:14Z

@ellenhp Oh, very happy to see you back Ellen and thanks for the review! I still have a few bugs to fix, but should be able to get this one done soon.

modules/minimp3/audio_stream_mp3.cpp

adamscott · 2024-01-11T14:10:00Z

scene/2d/audio_stream_player_2d.cpp

+	if (I) {
+		ParameterData &pd = I->value;
+		pd.value = p_value;
+		for (Ref<AudioStreamPlayback> &playback : stream_playbacks) {
+			playback->set_parameter(pd.path, pd.value);
+		}
+		if (p_value.is_null()) {
+			playback_parameters.erase(p_name);
+		}
+		return true;
+	}
+
+	return false;


I would use the return early pattern instead of a huge if.

Suggested change

if (I) {

ParameterData &pd = I->value;

pd.value = p_value;

for (Ref<AudioStreamPlayback> &playback : stream_playbacks) {

playback->set_parameter(pd.path, pd.value);

}

if (p_value.is_null()) {

playback_parameters.erase(p_name);

}

return true;

}

return false;

if (!I) {

return false;

}

ParameterData &pd = I->value;

pd.value = p_value;

for (Ref<AudioStreamPlayback> &playback : stream_playbacks) {

playback->set_parameter(pd.path, pd.value);

}

if (p_value.is_null()) {

playback_parameters.erase(p_name);

}

return true;

adamscott · 2024-01-11T14:11:53Z

scene/3d/audio_stream_player_3d.cpp

+	if (I) {
+		ParameterData &pd = I->value;
+		pd.value = p_value;
+		for (Ref<AudioStreamPlayback> &playback : stream_playbacks) {
+			playback->set_parameter(pd.path, pd.value);
+		}
+		if (p_value.is_null()) {
+			playback_parameters.erase(p_name);
+		}
+		return true;
+	}
+
+	return false;


Same return early comment here.

I think we will need to have a discussion about these things because I don't think we need to unify so much in style for this type of code. To me early return is not necessarily more readable always. It also makes more sense in a context like this one where all the 3 functions (_get _set _get_property list) work the same. I will do it for this PR, but I think we need to lax the review requirements a bit.

adamscott · 2024-01-11T14:12:20Z

scene/audio/audio_stream_player.cpp

+	if (I) {
+		ParameterData &pd = I->value;
+		pd.value = p_value;
+		for (Ref<AudioStreamPlayback> &playback : stream_playbacks) {
+			playback->set_parameter(pd.path, pd.value);
+		}
+		if (p_value.is_null()) {
+			playback_parameters.erase(p_name);
+		}
+		return true;
+	}
+
+	return false;


Same return early comment here.

reduz · 2024-01-13T13:53:34Z

Now the parameter is always null and can't be changed.

Ah shit, I thought It worked, let me test again 😆

reduz · 2024-01-13T13:58:26Z

Yeah nevermind, I was misled thinking it worked, can't do it like this. Guess the null will have to stay if unassigned, else the logic gets more hairy due to using checkable logic.

KoBeWi · 2024-01-13T15:12:25Z

Maybe you can handle it using _validate_property(). The null only appears after toggling the parameter, so you get a new value that can't be removed anymore.

reduz · 2024-01-13T15:23:35Z

@KoBeWi The problem is that this is very parameter dependant, and this null comes from a checkable parameter in the OGG/MP3 streams, other parameters may not necessarily be like this.
I could still add a hack in _validate_property and make a parameter not serializable if its null, but it feels too workaroundish and not sure if this is worth it.

KoBeWi · 2024-01-13T15:44:10Z

We'll get a report as soon as someone finds out a parameter in their scene that they can't remove.
EDIT:
Actually the parameter appears right after assigning the stream. Maybe a proper fix is tweaking the behavior of checkable properties, but that's out of scope of this PR.

Mickeon · 2024-01-13T16:30:41Z

The fact that null is saved as is now needs to be addressed before this gets merged, otherwise it's not going to leave a great impression, in my opinion. The feature would feel underbaked right away. There's many cases of properties saving when they don't need to, and I wouldn't want this to contribute to the pile, either.

KoBeWi · 2024-01-13T16:34:59Z

I could still add a hack in _validate_property and make a parameter not serializable if its null, but it feels too workaroundish and not sure if this is worth it.

Actually this should check if the value is equal to the default value defined in the parameter.

KoBeWi

Well, aside from the null issue, looks fine.
I'll try to address it when I update #87061.

MJacred

Follow-up from previous review

modules/vorbis/audio_stream_ogg_vorbis.cpp

Implements a way for audio stream playback to be configured via parameters directly in the edited AudioStreamPlayer[2D/3D]. Currently, configuring the playback stream is not possible (or is sometimes hacky as the user has to obtain the currently played stream, which is not always immediately available). This PR only implements this new feature to control looping in stream playback instances (a commonly requested feature, which was lost in the transition from Godot 2 to Godot 3). But the idea is that it can do a lot more: * If effects are bundled to the stream, control per playback instance parameters such as cutoff or resoance, or any other exposed effect parameter per playback instance. * For the upcoming interactive music PR (godotengine#64488), this exposes an easy way to change the active clip, which was not possible before. * For the upcoming parametrizable audio support (godotengine/godot-proposals#3394) this allows editing and animating audio graph parameters. In any case, this PR is required to complete godotengine#64488. Update modules/vorbis/audio_stream_ogg_vorbis.h Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com> Update modules/minimp3/audio_stream_mp3.h Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com> Update modules/minimp3/audio_stream_mp3.h Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com> Update modules/vorbis/audio_stream_ogg_vorbis.h Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com> Update doc/classes/AudioStream.xml Co-authored-by: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com>

reduz · 2024-01-16T09:32:42Z

Alright, should be ready to merge once it passes CI.

akien-mga · 2024-01-16T10:16:15Z

Thanks!

reduz · 2024-01-16T10:44:32Z

🎉

Thanks all involved for giving me feedback on this!

reduz requested review from a team as code owners December 23, 2023 16:41

Calinou added enhancement topic:audio labels Dec 23, 2023

Calinou added this to the 4.x milestone Dec 23, 2023

AThousandShips reviewed Dec 23, 2023

View reviewed changes

doc/classes/AudioStream.xml Outdated Show resolved Hide resolved

reduz force-pushed the audio-playback-parameters branch from f2d3ad8 to 8791576 Compare December 23, 2023 18:06

reduz force-pushed the audio-playback-parameters branch from 8791576 to 3b8ad0b Compare January 8, 2024 14:21

adamscott self-requested a review January 8, 2024 20:43