Audio Workers: number of input/output channels and inputs/outputs

[olivierthereaux]

Originally reported on W3C Bugzilla ISSUE-17534 Mon, 18 Jun 2012 11:27:42 GMT
Reported by Marcus Geelnard (Opera)
Assigned to

The JavaScriptAudioNode does not have the ability to dynamically change its number of input/ouptut channels after creation.

This makes it impossible to re-implement nodes such as AudioGainNode (depends on number of input channels) and ConvolverNode (depends on number of AudioContext output channels - according to [1]).

[1] https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#Convolution-reverb-effect

[olivierthereaux]

Original comment by Olivier Thereaux on W3C Bugzilla. Fri, 26 Jul 2013 15:02:50 GMT

Updating issue title to reflect the change of name from JavaScriptAudioNode to
ScriptProcessorNode

[agoode]

What is the next step here? Is it as simple as just proposing some changes to the specification?

[olivierthereaux]

@agoode this issue has not been a high priority as there has been a number of other issues to be fixed first to make ScriptProcessorNode actually usable, but yes - a PR against the spec would be a good way to get this one forward.

[cwilso]

Would be good to discuss whether we need this feature for Audio Workers. We probably do. Sigh.

[joeberkovitz]

Does this go beyond simply stating that the AudioProcessEvent can bear a different number of channels over the lifetime of the AudioWorkerGlobalScope?

[cwilso]

Yes. The AudioWorker needs to be able to CONTROL the number of output channels (in order to replicate channelMerger).

[joeberkovitz]

Idea: Could there be a function in AudioWorkerGlobalScope that can be used to update the number of output channels in future AudioProcessEvents, and returns that many usable output buffers for use in the current event?

[cwilso]

For the number of channels, I think we can simply update an attribute and state that the hosting code will use that as a way to determine how many output buffers to provide (input buffers will be determined by the channelcount and channelmode).

Input and output connections are going to be harder.

[joeberkovitz]

But that approach to dynamic channel count doesn't work when the number of channels is dynamically determined inside onaudioprocess, does it?

Also, don't understand what connections have to do with this. Standard upmixing/downmixing should apply, yes?

[cwilso]

You would have to determine the number of channels before onaudioprocess is called, yes.

Hmm, thought that would be okay if we got an onconnect notification, but actually no - it's going to have to be dynamic inside (since it may depend on number of channels). That means we're going to need a more complex system for requesting more/fewer channels.

Upmixing and downmixing only control the number of channels inside a connection. Nodes can have multiple inputs and outputs, and multiple channels inside each of them.

[joeberkovitz]

Agreed now that the number of input/output ports is a tricky problem. I do know how up/downmixing works, but I went astray in response to your mention of the number of "connections". You clarified the issue as concerning the number of inputs/outputs, which I'd forgotten about also. Ugh. I guess AudioProcessEvent needs an upgrade, at the least.

Aren't the number of inputs and outputs for all the built-in nodes constant? If so, could these just be new initialization parameters when an AudioWorkerNode is created?

About the channel count: would it work if the node could change its channel input/output counts in response to an event that's dispatched just before the next sample block is processed, which passes in the incoming channel counts for each of the inputs? I'm not sure if this is what you meant by "onconnect", but presumably it would be an event that occurs when there's some change in the number of live inputs to a node.

[cwilso]

Hmm, yes, on investigation the number of inputs and number of outputs is given on creation, and remains constant. (The number of channels in the inputs and outputs does NOT need to remain constant, however, and that's still gonna make this tricky.)

The merger needs to know 1) how many inputs are connected (not just how many were created, but how many have active connections) and 2) how many channels in each of those connected inputs. Those variables drive how many output channels it will need. Unfortunately, I think this is going to be a two-step process, because merger is very dynamic in how this channel mapping occurs.

However, given that I think merger and splitter are the only nodes with this kind of dynamic channel changing, I think it might be possible to make this a very special case, with a separate route (that most Worker nodes won't need to worry about at all).

[joeberkovitz]

TPAC: we will proceed on the assumption that the spec editors will find a solution to this.

[joeberkovitz]

I believe this is covered as part of the resolution to #113. Closing