Skip to content
Permalink
Browse files
[GTK][GStreamer] Web Audio - Media element source - Audio is cracking.
https://bugs.webkit.org/show_bug.cgi?id=196293

Patch by Philippe Normand <pnormand@igalia.com> on 2021-04-06
Reviewed by Xabier Rodriguez-Calvar.

The provider client might request samples faster than the current clock speed, so this sink
should process buffers as fast as possible. The cracks were consequence of the audio sink of
the AudioDestination starving off.

* platform/audio/gstreamer/AudioSourceProviderGStreamer.cpp:
(WebCore::AudioSourceProviderGStreamer::handleNewDeinterleavePad):

Canonical link: https://commits.webkit.org/236169@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@275512 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
philn authored and webkit-commit-queue committed Apr 6, 2021
1 parent d493665 commit 861f571fa82a0a45293cc8e546b9ef5e7dbbca4d
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 7 deletions.
@@ -1,3 +1,17 @@
2021-04-06 Philippe Normand <pnormand@igalia.com>

[GTK][GStreamer] Web Audio - Media element source - Audio is cracking.
https://bugs.webkit.org/show_bug.cgi?id=196293

Reviewed by Xabier Rodriguez-Calvar.

The provider client might request samples faster than the current clock speed, so this sink
should process buffers as fast as possible. The cracks were consequence of the audio sink of
the AudioDestination starving off.

* platform/audio/gstreamer/AudioSourceProviderGStreamer.cpp:
(WebCore::AudioSourceProviderGStreamer::handleNewDeinterleavePad):

2021-04-06 Philippe Normand <pnormand@igalia.com>

[GStreamer] Heap allocation exceptions
@@ -291,8 +291,9 @@ void AudioSourceProviderGStreamer::handleNewDeinterleavePad(GstPad* pad)
// A new pad for a planar channel was added in deinterleave. Plug
// in an appsink so we can pull the data from each
// channel. Pipeline looks like:
// ... deinterleave ! appsink.
GstElement* sink = gst_element_factory_make("appsink", nullptr);
// ... deinterleave ! queue ! appsink.
auto* queue = gst_element_factory_make("queue", nullptr);
auto* sink = gst_element_factory_make("appsink", nullptr);

static GstAppSinkCallbacks callbacks = {
nullptr,
@@ -305,15 +306,19 @@ void AudioSourceProviderGStreamer::handleNewDeinterleavePad(GstPad* pad)
{ nullptr }
};
gst_app_sink_set_callbacks(GST_APP_SINK(sink), &callbacks, this, nullptr);
g_object_set(sink, "async", FALSE, nullptr);
// The provider client might request samples faster than the current clock speed, so this sink
// should process buffers as fast as possible.
g_object_set(sink, "async", FALSE, "sync", FALSE, nullptr);

auto caps = adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, static_cast<int>(gSampleBitRate),
"channels", G_TYPE_INT, 1, "format", G_TYPE_STRING, GST_AUDIO_NE(F32), "layout", G_TYPE_STRING, "interleaved", nullptr));
gst_app_sink_set_caps(GST_APP_SINK(sink), caps.get());

gst_bin_add(GST_BIN_CAST(m_audioSinkBin.get()), sink);
gst_bin_add_many(GST_BIN_CAST(m_audioSinkBin.get()), queue, sink, nullptr);

auto sinkPad = adoptGRef(gst_element_get_static_pad(sink, "sink"));
gst_element_link(queue, sink);

auto sinkPad = adoptGRef(gst_element_get_static_pad(queue, "sink"));
gst_pad_link_full(pad, sinkPad.get(), GST_PAD_LINK_CHECK_NOTHING);

GQuark quark = g_quark_from_static_string("peer");
@@ -335,6 +340,7 @@ void AudioSourceProviderGStreamer::handleNewDeinterleavePad(GstPad* pad)
return GST_PAD_PROBE_OK;
}, this, nullptr);

gst_element_sync_state_with_parent(queue);
gst_element_sync_state_with_parent(sink);
}

@@ -351,9 +357,14 @@ void AudioSourceProviderGStreamer::handleRemovedDeinterleavePad(GstPad* pad)
if (!sinkPad)
return;

auto sink = adoptGRef(gst_pad_get_parent_element(sinkPad));
auto queue = adoptGRef(gst_pad_get_parent_element(sinkPad));
auto srcPad = adoptGRef(gst_element_get_static_pad(queue.get(), "src"));
auto sinkSinkPad = adoptGRef(gst_pad_get_peer(srcPad.get()));
auto sink = adoptGRef(gst_pad_get_parent_element(sinkSinkPad.get()));
gst_pad_unlink(srcPad.get(), sinkSinkPad.get());
gst_element_set_state(queue.get(), GST_STATE_NULL);
gst_element_set_state(sink.get(), GST_STATE_NULL);
gst_bin_remove(GST_BIN_CAST(m_audioSinkBin.get()), sink.get());
gst_bin_remove_many(GST_BIN_CAST(m_audioSinkBin.get()), queue.get(), sink.get(), nullptr);
}

void AudioSourceProviderGStreamer::deinterleavePadsConfigured()

0 comments on commit 861f571

Please sign in to comment.