OfflineAudioContext progressive rendering

[mudcube]

Has there been consideration to adding an onProgress event to the OfflineAudioContext rendering? It seems this would be very useful when mixing down larger songs to allow users to know how long the process is going to take.

[joeberkovitz]

Also consider this comment by Srikumar (originally in #69) that feels very significant to me:

Synchronous graph manipulation is, I'd say, a must have in the offline case. If I want to mix down a 5 minute composition with 8 tracks with 1 voice in each track triggered every second (not too taxing a scenario for the realtime composer case), that would need 2400 nodes to be pre-created ... and this doesn't count effects nodes. Permitting the graph to be changed within a script node would provide a way to instantiate these voices just in time.

If that isn't possible, it would help to add a separate synchronous event that can be used for this purpose, that would also be useful for tracking progress of an offline context.

[joeberkovitz]

A couple more points here:

1. Incremental construction for online rendering will likely already have been implemented in setInterval/setTimeout handlers that build nodes needed for a window of AudioContext time in the near future. To conveniently do this for offline rendering, it would be great if the "progress" events here behaved similarly but with respect to offline rendering time -- i.e., they fire when a given time interval in rendering time has elapsed, instead of an interval in clock time.
2. To allow incremental construction of audio nodes for offline rendering in a handler for such an event, without the offline context "racing ahead" in its own thread, it's necessary to pause the rendering while the handler is active.

[joeberkovitz]

TPAC resolution: Introduce a way to tell OfflineAudioContext to pause automatically at some predetermined interval in context rendering time units. The resulting state change event can be used by applications to notify when the context has paused. This also serves to notify of progress. Explicit resume is required to make the context start resuming again.

[cwilso]

Per #21, we should enable a way to get partial output too.

[joeberkovitz]

#473 (now closed) proposed that another way of dealing with this issue is to have a single event callback on a context (either realtime or offline) that can be fired at some designated audio context time. It could either replenish a realtime graph that continues to run, or [synchronously] replenish an offline graph that is paused during the event dispatch.

[hoch]

For the moment, I am exploring with few ideas on pausing OfflineAudioContext, which turned out to be super-useful for the API testing.

(1) onpause event: although it is an old pattern, but it is great for the recurring routine with an interval.

var context = new OfflineAudioContext(2, 441000, 44100);

context.pauseAtTime(2.0);

context.onpause = function (event) {
  // change your graph or do something with the partial buffer.
  var channelData0 = event.renderedBuffer.getChannelData(0);
  // ...and schedule the next pause.
  context.pauseAtTime(event.playbackTime + 2.0);
}

context.startRendering();

(2) pauseAtTime() promise pattern: this pattern is useful when there are specific tasks for multiple time points.

var context = new OfflineAudioContext(2, 441000, 44100);

context.pauseAtTime(2.0).then(function (buffer, playbackTime) {
  // do stuffs here (1).
});

context.pauseAtTime(6.5).then(function (buffer, playbackTime) {
  // do stuffs here (2).
});

context.startRendering();

In my opinion, we need both patterns - but probably (hopefully) we can have something much better pattern than these.