Add a wasm-bindgen based generic WebAudio backend.

[dpeckett]

Add a generic wasm-bindgen based WebAudio backend. Uses a sample scheduling strategy based on a set of interleaved AudioBufferSourceNode's which are rotated in and out as they finish playback. A quick port of the beep example appears to play stutter free on Firefox and Chrome.

[mitchmindtree]

Thanks a lot @dpeckett, at a glance this looks good! I'll give this a closer look and test in the next day or two and report back 👍

[mitchmindtree]

Hi @dpeckett! I'm just having a go at getting this backend running. Should it be possible to do something like this?:

cargo install cargo-web
cargo web start --features "wasm-bindgen" --example beep

When I give it a go, it runs a local server with my application at '/beep.js' but I'm unable to hear anything (in firefox at least).

Just thought I'd check to see if you could share your workflow before diving in deeper!

[dpeckett]

It seems most browsers implement a security policy requiring that audio playback be initiated by a user event. Eg a button press. I tested the backend by hacking something together based on the example here: https://github.com/rustwasm/wasm-bindgen/tree/master/examples/webaudio

[ryanisaacg]

I would also note that I don't think cargo-web and wasm-bindgen are compatible?

[ishitatsuyuki]

Hi there! I'm currently working on a demo of this PR using wasm-pack, but apparently the implementation is giving inconsistent output.

I ported the beep example, and the output is either the expected sine wave or a clipped one (probably multiple sine wave stacked together). Is this a known bug in which case you can give some insights? Otherwise I'll dig into this later.

[dpeckett]

@ishitatsuyuki this isn't something I ran into in my experiments. Can you share your beep port?

[ishitatsuyuki]

@dpeckett I uploaded my branch here: https://github.com/ishitatsuyuki/cpal/tree/dpeckett-webaudio-poc

I first tried with the current PR revision, then rebased. Both gave the same glitchy results.

[ishitatsuyuki]

okay, I just realized that I messed up the webpack configuration so that the init script was being executed twice (with isolated contexts). Things should be working now, although the example did gave some timing glitches. I don't know if it would be solved when running with release.

[dpeckett]

Excellent, good to hear that worked out. Definitely worth trying out a release build, in initial tests I saw some glitchiness with scheduling the first buffer but once running it appeared stable. This was on linux amd64 firefox/chrome with a release build. When are you hearing the glitchiness? At initial scheduling or during playback?

[ishitatsuyuki]

I run on the same platform as you and yes, I think the glitches occur only when the stream starts.

[dpeckett]

The early stream underruns are definitely the weakest part of the current approach. There's a constant in the code which allows for a small initial delay. It's currently set at 25ms but perhaps something else can be investigated or the value increased.

[mitchmindtree]

Thanks for linking your demo @ishitatsuyuki, it made it a lot easier to test this PR!

I had some unrelated npm issues on my nixos machine, but my housemate was able to get it building and running on his ubuntu machine quite easily after installing a bunch of dependencies npm reported as missing.

I'm happy for this to land if you both are @ishitatsuyuki, @dpeckett? Perhaps we can also open issues as reminders to investigate solutions for the arbitrary initial delay and input stream support and address them in future PRs.

[mitchmindtree]

Oh while I remember, @dpeckett could you also update the CHANGELOG?

[expenses]

Hey! I'm a big fan of the work you've done for this PR, however I am finding that the time delay makes it unsuitable for games at present. Is there a way to avoid having this delay? Thanks!

[dpeckett]

Hey! I'm a big fan of the work you've done for this PR, however I am finding that the time delay makes it unsuitable for games at present. Is there a way to avoid having this delay? Thanks!

Ultimately all audio subsystems will be subject to some amount of latency. The latency in this implementation can definitely be tightened up but I still don't see it being a great fit for realtime audio effects (due to weak integration between WASM and the web audio API).

Now this isn't to say WASM and the web audio API couldn't be used for effects, it would just need to be served by a radically different architecture. Eg. Hand over all sample processing to the JS world, and use WASM to load and trigger effects/samples (WASM no longer in the sample hot path). This is very different to cpal but if someone gets interested enough, this is probably something that should exist.

[ryanisaacg]

How long of a delay can be expected on average?

[dpeckett]

How long of a delay can be expected on average?

I'd have to benchmark things thoroughly before I could give a solid number (which would be a significant effort) but just on gut feel I'd expect latency on the order of atleast 100ms with the current approach.

[ishitatsuyuki]

Due to this WIP, the lower bound of delay in this impl is 0.33s.

[dpeckett]

Due to this WIP, the lower bound of delay in this impl is 0.33s.

The 100ms ballpark figure is a guess on the theoretical lower bound from my early experiments but I'm sure someone might be able to improve on it. Just want to be clear about ballpark latency expectations.

[mitchmindtree]

I'm going to rebase this onto master, update the changelog, add a timestamp implementation and open a new PR so we can land this and make sure it doesn't get left behind!

For those interested in latency, @JoshuaBatty has been doing some research into a buffer size API that should be compatible with all platforms and will open a PR to get some initial feedback soon (Edit: #401). Adding user configurable buffer size support might be at least one step towards allowing for tuning latency, though as @dpeckett mentions there are still other sources of latency to look into.

[dpeckett]

I'm going to rebase this onto master, update the changelog, add a timestamp implementation and open a new PR so we can land this and make sure it doesn't get left behind!

Thanks Mitch, I haven't worked on the cpal codebase in a while, so this is a great assistance.

For those interested in latency, @JoshuaBatty has been doing some research into a buffer size API that should be compatible with all platforms and will open a PR to get some initial feedback soon (Edit: #401). Adding user configurable buffer size support might be at least one step towards allowing for tuning latency, though as @dpeckett mentions there are still other sources of latency to look into.

Exposing latency information and adding some way to tune buffer sizes is definitely the best way forward with this as it stands.

[mitchmindtree]

Closed via #406.