Unclear behavior of sources scheduled at fractional sample frames

[joeberkovitz]

The spec is silent on the consequences of scheduling a source at a time that does not fall on an exact start time of some sample frame, where such frames are imposed by the AudioContext's overall sample rate.

Experimentation suggests that current browser implementations only schedule sources on an exact sample frame boundary. In particular this represents a problem for stitching together AudioBufferSourceNodes to create seamless output when the playback rates and sample rates of these sources mean that the underlying AudioBuffer frames are not aligned with the prevailing AudioContext frames.

While subsample scheduling may seem fussy, there is an important use case (see issue cited below). Let's also note that AudioBufferSourceNodes are already dealing with cases where the underlying AudioBuffer has a different sample rate from the AudioContext. Thus dynamic resampling of audio buffers is an integral part of all existing implementations. On the other hand, one would not want small errors in scheduling times to force the engine to perform expensive-but-inaudible resampling just to reflect start times that lie slightly off of an exact sample frame. So perhaps this behavior needs to be controllable by developers.

A prior issue concerning lack of support for continuous, seamless audio appears to be related:

#265

[bjornm]

Thanks for looking into this. I'd appreciate if you also looked into the loop start and end points for the AudioBufferSourceNode - subsample precision seems kind of unnecessary :)

[joeberkovitz]

@bjornm I've recorded that issue as #336

FWIW I agree that subsample precision looping seems unnecessary. Looping rounded to exact sample frames is already seamless and glitch-free, regardless of the playback rate and sample rate of the buffer. It's a very different issue from this one.

[karlt]

Gecko 30 should do better with buffer sample rates differing from the context
sample rate. The approach used is to round the start time to the nearest
fraction of the context sample period given by
gcd(buffer rate, context rate)/(buffer rate).

That's just what worked well and didn't add any efficiency costs with the
speex resampler in Gecko, but not necessarily something that should be
specified.

Even with buffer rate matching the context rate, unlucky positioning can still
end up with problems due to start times having double representation and so
may round up and down for different buffers. This would not happen if start
times are based on an integer number of buffer sample periods from a time
returned by currentTime, but an offset that is an odd multiple of 5ms will
fall exactly midway between 44100 Hz sample points.

[joeberkovitz]

TPAC resolution: edit spec to stipulate that all sources are always scheduled to occur on the exact sample frame nearest to the requested time.