Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Kira tracks #12

Open
SolarLiner opened this issue May 6, 2024 · 0 comments
Open

Integrate Kira tracks #12

SolarLiner opened this issue May 6, 2024 · 0 comments
Labels
Exploration R&D on possible changes related to a feature or a bug Feature New additions to the API surface
Milestone

Comments

@SolarLiner
Copy link
Owner

Summary

Integrate tracks as entities in Bevy.

Implementation proposal

Tracks

Similarly to how audio sources are currently integrated, tracks can be added in two steps:

  1. A first struct which details the track setup is added to the ECS by the user through a TrackBundle.
  2. A ECS system reacts to additions of this component to create the Kira track with the specified properties.
    Due to a limitation with Kira, tracks cannot be updated after they're created, so the original struct cannot be used to update the track live (needs Kira issue to change this)

Track handles should be wrapped in a newtype component to let users change the track after it has been created.

Effects

Tracks in Kira have an effects rack, a list of effects that process the sound coming into the track sequentially. Support for effects is a highly requested feature, so the API for it should be as intuitive as possible.

Effects should ideally be thought of as entities as well, which simplifies access from systems. They need to be attached to a track, which eventually can become a relationship once those land in future versions of Bevy; however for now, the only feature we can reliably use is the parenting relationship system; so for now, effects can be implemented as being children entities of the track entity they're on.

Integration of effects into the ECS is done is much of the same way as audio sources. Since there are multiple types of effects, each strongly typed, there should be a generic plugin that takes care of the integration.

The user would specify a settings component through an EffectBundle, which would then apply the effect to the parented track. An EffectControl component is then made available once the effect is added to the track, making it available to be controlled by other systems.

Another downside of the approach is that effect ordering is not explicitly given; insertion order can be recovered from the entity ID, which should be monotonically increasing in terms of its Ord implementation. This is fine, as there are no ways to either reorder effects or any way to insert any post-creation (see Limitations below).

Limitations

Tracks and effects in Kira cannot be "structurally changed" after being created, ie. you cannot add or remove tracks, change their routing, or add or reorder effects after the track has been created. This means that all track-related setup needs to happened within the same tick, which is unfortunate and the path forward most likely warrants asking for more runtime control from Kira.

Possible alternatives

"Render graph"-like setup

Another possibility is to provide a plugin build-time API for creating tracks and effects. This would disallow adding new tracks after the fact, but lends itself to Kira's way of working better. After build, the mixer structure is frozen, and one can only interact with tracks from a Mixer resource which owns all track and effect handles. This also means that audio systems modifying tracks and effects essentially cannot run in parallel, but also simplifies the implementation.

Effects as part of the track

The track can "own" the effects, which simplifies integration and setup, but means all queries changing effects need to have exclusive access over audio track components, which limits the parallelization the scheduler can do over audio-centric systems. It allows a more explicit ordering of effects though.

@SolarLiner SolarLiner added Feature New additions to the API surface Exploration R&D on possible changes related to a feature or a bug labels May 6, 2024
@SolarLiner SolarLiner added this to the 0.3 milestone May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Exploration R&D on possible changes related to a feature or a bug Feature New additions to the API surface
Projects
Development

No branches or pull requests

1 participant