Sound: Various issues under Windows

[ndarilek]

Not sure how to title this issue, and I'm not sure if my use of Windows is relevant. I've been trying to use rg3d-sound in Bevy, and have hit an odd issue where my footstep sound is slightly delayed every few seconds. I also, very occasionally, get an issue where sound glitches and can be heard panned slightly differently in the stereo field, as if the source parameters aren't quite synced.

I thought I'd try creating a reproduction, which I have here. But while I can't reproduce the delay, I've found a few issues that may be related.

Essentially, I took your play_sound example, subbed in my footstep sound, and added a loop { ... } block. I need to start/stop my footstep sound more or less quickly based on step length, so I need to control the looping. Here's what I discovered:

• I have the loop set to sleep for 5 seconds, with a println! at the beginning. The print is called correctly, but the sound still seems to loop quickly despite me explicitly setting it not to.
• If I use from_millis(5000) for the sleep, the behavior changes. After 20-30 seconds, I start getting audio artifacts similar to the ones I occasionally experience, where the source sounds slightly out of sync.

It's quite possible that I'm misusing the library here, but it seems like it's misbehaving in ways that don't make sense. I don't understand why the source doesn't play once per 5 seconds, nor do I understand the odd audio artifacts here. I might expect performance degradation at higher speeds as the mutex deadlocks, but not at 1/5 seconds.

Am I doing something obviously wrong? Full disclosure: haven't had my coffee yet. :)

Thanks!

[mrDIMAS]

Hi! You locking mutex for 5 seconds and mixer thread just does not have enough time to mix everything and send data to sound device. Here is your loop:

loop {
	println!("Looping");
	let mut context = context.lock().unwrap();
	let source = context.source_mut(source_handle);
	source.play();
	// This introduces audio artifacts after a while.
	// Use `from_secs(5)` and these go away for me.
	thread::sleep(Duration::from_millis(5000));
}

and here is correct version:

loop {
	println!("Looping");
	// This scope ensures that mutex will be released before calling thread::sleep.
	// Alternatively you can std::mem::drop(context) after source.play(); which will
	// also release the lock.
	{
		let mut context = context.lock().unwrap();
		let source = context.source_mut(source_handle);
		source.play();
	}
	// This introduces audio artifacts after a while.
	// Use `from_secs(5)` and these go away for me.
	thread::sleep(Duration::from_millis(5000));
}

So, general rule is to lock mutex as short time as possible, otherwise mixer thread will just wait and sound will be incorrect.

[ndarilek]

Oh, duh. Sorry for the silly mistake.

Now, with this code, I'm able to trigger the odd delays. I would expect this footstep sound to play rhythmically. But, on my system, it doesn't. There are odd delays every few footsteps, and the pattern doesn't repeat. To be clear, I don't expect it to cleanly loop. I just expect it to sound rhythmic, but instead it is more staggered.

Not sure if play is meant to restart the source, which is why I'm stopping it.

Am I doing something wrong here as well? Thanks.

use std::{thread, time::Duration};

use rg3d_sound::{
    buffer::{DataSource, SoundBuffer},
    context::Context,
    pool::Handle,
    source::{generic::GenericSourceBuilder, SoundSource},
};

fn main() {
    // Initialize new sound context with default output device.
    let context = Context::new().unwrap();

    // Load sound buffer.
    let footstep_buffer =
        SoundBuffer::new_generic(DataSource::from_file("footstep.wav").unwrap()).unwrap();

    // Create generic source (without spatial effects) using that buffer.
    let source = GenericSourceBuilder::new(footstep_buffer)
        .with_looping(false)
        .build_source()
        .unwrap();

    // Each sound sound must be added to context, context takes ownership on source
    // and returns pool handle to it by which it can be accessed later on if needed.
    let source_handle: Handle<SoundSource> = context.lock().unwrap().add_source(source);
    // This plays the sound way more than once every 5 seconds.
    loop {
        {
            let mut context = context.lock().unwrap();
            let source = context.source_mut(source_handle);
            source.stop().unwrap();
            source.play();
        }
        thread::sleep(Duration::from_millis(500));
    }
}

[mrDIMAS]

Here is the picture that should clarify what is going on under the hood:

So there are two threads - one is user thread, and second is mixer thread. Mixer thread mix samples into audio buffer and sends data to output device, the time that send procedure could take is undefined, it depends on the driver, OS, and other stuff. So perfect looping could be achived only by setting .with_looping(true) on a sound source. By doing this, looping happens in mixer thread and it is able to do perfect loop. Please note that backend implemented using ring buffer and by phrase "mixing another portion" on the picture I mean that we mixing a portion of samples for the next iteration, not the current one.

[ndarilek]

Sorry, I'm blind and can't see the picture. But to be clear, I'm not interested in looping a sound. If someone is running, for instance, there would be less time between their individual footsteps, and you wouldn't want a previous footstep overlapping the next one, nor would you want the previous one playing where they were. What is the correct way to handle that with your engine? Should I be creating a new footstep sound and stopping the previous one? I'd hoped to just reuse the source, move it and restart it. Thanks.

[mrDIMAS]

Oh, sorry. What I'm doing in my shooter is that I'm creating new source per footstep. Also the engine can automatically handle lifetime of a source for such temporary sound sources. So you need to create a source with .with_play_once(true) and just play it and forget about it - it will die automatically when it ends. Creating a source is cheap, however you should ensure that amount of simultaneously playing sources is relatively low (soft cap here is about 64 sources), otherwise mixer probably will take large amount of time and sound will stutter.

Other way of implementing this is to use multiple sources with different buffers - each buffer should have slightly different version of footstep to sound more naturally. In this case you should stop currently playing source, and then play a random one.

I'm not a sound designer and don't know all the tricks that will result in natural sound, just relying on what I hear.

[ndarilek]

Got it, so seems this is working as designed. Any chance you might eventually support more responsive stopping/restarting of sources? Or should this be closed?

Having to create new sources vs. controlling an existing one would seem to make some types of problems more difficult. It's also a bit more verbose, since instead of just reusing this footstep source, I'd have to create a new one each time.

Thanks for your help.

[mrDIMAS]

I can't imagine a situation in game where such precision is needed, in case of footsteps small variation is good actually, in other cases such small delay is ok too for me 🤔 .

[ndarilek]

I think you're technically right in that the footstep delay isn't bad from an aesthetic perspective, as it makes them sound more natural. But I think those should be designed in, rather than done that way by necessity. Anyhow, thanks for your work on this project. I respect that it meets your needs, so I'll let you decide whether or not to close this. I also appreciate the quick help!