Audio playback rate sped up for few seconds when using createMediaEle…

…mentSource

https://bugs.webkit.org/show_bug.cgi?id=239696
<rdar://problem/92569332>

Reviewed by Eric Carlson.

We added a way for AudioSampleDataSource to reduce latency due to its audio buffer by reading the audio buffer faster
until the audio buffer size decreases to a reasonnable value.
The reasonnable buffer size was computed in terms of multiple of 10ms as RTC sources are pushing 10 or 20 millisecond chunks.
Other sources may push much bigger chunks (100 milliseconds for instance), in which case it does not make real sense to try to reduce the delay below it.
To prevent this, we will decrease the buffer size if it is above 100 milliseconds (as previously) AND the buffer size is 4 times the size of the pushed audio chunk.
4 times the size of the pushed audio chunk is 40 ms for RTC sources which is a reasonnable buffer size.

Manually tested with https://webaudioapi.com/samples/audio-tag/.

* platform/audio/cocoa/AudioSampleDataConverter.h:
* platform/audio/cocoa/AudioSampleDataConverter.mm:
(WebCore::AudioSampleDataConverter::updateBufferedAmount):
* platform/audio/cocoa/AudioSampleDataSource.mm:
(WebCore::AudioSampleDataSource::pushSamplesInternal):

Canonical link: https://commits.webkit.org/250579@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@294221 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebCore/ChangeLog

@@ -1,3 +1,26 @@
+2022-05-15  Youenn Fablet  <youenn@apple.com>
+
+        Audio playback rate sped up for few seconds when using createMediaElementSource
+        https://bugs.webkit.org/show_bug.cgi?id=239696
+        <rdar://problem/92569332>
+
+        Reviewed by Eric Carlson.
+
+        We added a way for AudioSampleDataSource to reduce latency due to its audio buffer by reading the audio buffer faster
+        until the audio buffer size decreases to a reasonnable value.
+        The reasonnable buffer size was computed in terms of multiple of 10ms as RTC sources are pushing 10 or 20 millisecond chunks.
+        Other sources may push much bigger chunks (100 milliseconds for instance), in which case it does not make real sense to try to reduce the delay below it.
+        To prevent this, we will decrease the buffer size if it is above 100 milliseconds (as previously) AND the buffer size is 4 times the size of the pushed audio chunk.
+        4 times the size of the pushed audio chunk is 40 ms for RTC sources which is a reasonnable buffer size.
+
+        Manually tested with https://webaudioapi.com/samples/audio-tag/.
+
+        * platform/audio/cocoa/AudioSampleDataConverter.h:
+        * platform/audio/cocoa/AudioSampleDataConverter.mm:
+        (WebCore::AudioSampleDataConverter::updateBufferedAmount):
+        * platform/audio/cocoa/AudioSampleDataSource.mm:
+        (WebCore::AudioSampleDataSource::pushSamplesInternal):
+
 2022-05-15  Wenson Hsieh  <wenson_hsieh@apple.com>
 
         Right click > "Open Link" should trigger a navigation action with WKNavigationTypeLinkActivated

Source/WebCore/platform/audio/cocoa/AudioSampleDataConverter.h

@@ -42,7 +42,7 @@ class AudioSampleDataConverter {
     ~AudioSampleDataConverter();
 
     OSStatus setFormats(const CAAudioStreamDescription& inputDescription, const CAAudioStreamDescription& outputDescription);
-    bool updateBufferedAmount(size_t currentBufferedAmount);
+    bool updateBufferedAmount(size_t currentBufferedAmount, size_t pushedSampleSize);
     OSStatus convert(const AudioBufferList&, AudioSampleBufferList&, size_t sampleCount);
     size_t regularBufferSize() const { return m_regularBufferSize; }
     bool isRegular() const { return m_selectedConverter == m_regularConverter; }

Source/WebCore/platform/audio/cocoa/AudioSampleDataConverter.mm

@@ -72,13 +72,13 @@
     return noErr;
 }
 
-bool AudioSampleDataConverter::updateBufferedAmount(size_t currentBufferedAmount)
+bool AudioSampleDataConverter::updateBufferedAmount(size_t currentBufferedAmount, size_t pushedSampleSize)
 {
     if (currentBufferedAmount) {
         if (m_selectedConverter == m_regularConverter) {
             if (currentBufferedAmount <= m_lowBufferSize)
                 m_selectedConverter = m_lowConverter;
-            else if (currentBufferedAmount >= m_highBufferSize)
+            else if (currentBufferedAmount >= m_highBufferSize && currentBufferedAmount >= 4 * pushedSampleSize)
                 m_selectedConverter = m_highConverter;
         } else if (m_selectedConverter == m_highConverter) {
             if (currentBufferedAmount < m_regularLowBufferSize)

Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm

@@ -129,7 +129,7 @@
     int64_t offset = 0;
     const AudioBufferList* sampleBufferList;
 
-    if (m_converter.updateBufferedAmount(m_lastBufferedAmount)) {
+    if (m_converter.updateBufferedAmount(m_lastBufferedAmount, sampleCount)) {
         m_scratchBuffer->reset();
         m_converter.convert(bufferList, *m_scratchBuffer, sampleCount);
         auto expectedSampleCount = sampleCount * m_outputDescription->sampleRate() / m_inputDescription->sampleRate();