New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GStreamer][WebCodecs] Video encoding and decoding support #8399
Conversation
I wasn't able to make style-check happy. This is too cryptic for me: ERROR: /var/home/phil/WebKit/LayoutTests/platform/glib/TestExpectations:1152: More specific entry for imported/w3c/web-platform-tests/webcodecs/videoFrame-texImage.any.worker.html on line LayoutTests/platform/glib/TestExpectations:1152 overrides line LayoutTests/platform/glib/TestExpectations:1151. [test/expectations] [5] |
EWS run on previous version of this PR (hash e3453ea)
|
return VideoFrameGStreamer::create(WTFMove(sample), presentationSize); | ||
} | ||
|
||
RefPtr<VideoFrame> VideoFrame::createBGRA(Span<const uint8_t> span, size_t width, size_t height, const ComputedPlaneLayout& plane, PlatformVideoColorSpace&& colorSpace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe you can rework these two functions to use the same code but a flag variable? Am I missing any other difference? Not big deal anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can refactor this with a macro.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not a big fan of macros, TBH
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Too bad ;)
EWS run on previous version of this PR (hash 2dc05e0)
|
I can reproduce the webrtc/vp9.html crash in release (bt useless) but not in debug, so... π€·π» |
EWS run on previous version of this PR (hash 33a3aae)
|
Actually, this is due to the llvm15 update! The test also crashes in main for builds made with clang15. Gonna revert that one. |
EWS run on previous version of this PR (hash cfbab24)
|
GstCaps* filter = nullptr; | ||
GRefPtr<GstCaps> caps; | ||
if (m_outputCaps) | ||
caps = m_outputCaps.get(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need to keep the .get()
? I think you don't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, the way it was before, I think it was leaking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch!
GstCaps* filter = nullptr; | ||
GRefPtr<GstCaps> caps; | ||
if (m_inputCaps) | ||
caps = m_inputCaps.get(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto
EWS run on current version of this PR (hash 19f922e)
|
19f922e
to
93ae0b1
Compare
https://bugs.webkit.org/show_bug.cgi?id=249972 Reviewed by Xabier Rodriguez-Calvar. The video encoder makes use of the WebRTC encoder. The video decoder relies on the platform decoder previously probed through the registry scanner. In order to have integrate the encoding and decoding at the lowest level possible we make use of a new component called GstElementHarness, so we have full control on the input and output workflow of the encoder and decoder. * LayoutTests/platform/glib/TestExpectations: * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_h264-expected.txt: Added. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp8-expected.txt: Added. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp9-expected.txt: Added. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-createImageBitmap.any-expected.txt: Added. * LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-drawImage.any-expected.txt: Added. * Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml: * Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp: (WebCore::WebCodecsVideoFrame::create): * Source/WebCore/platform/GStreamer.cmake: * Source/WebCore/platform/VideoDecoder.cpp: (WebCore::VideoDecoder::createLocalDecoder): * Source/WebCore/platform/VideoEncoder.cpp: (WebCore::VideoEncoder::createLocalEncoder): * Source/WebCore/platform/VideoFrame.cpp: (WebCore::VideoFrame::paintInContext): * Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.cpp: Added. (WebCore::gstWorkQueue): (WebCore::GStreamerInternalVideoDecoder::create): (WebCore::GStreamerInternalVideoDecoder::postTask): (WebCore::GStreamerInternalVideoDecoder::close): (WebCore::GStreamerVideoDecoder::create): (WebCore::GStreamerVideoDecoder::GStreamerVideoDecoder): (WebCore::GStreamerVideoDecoder::~GStreamerVideoDecoder): (WebCore::GStreamerVideoDecoder::decode): (WebCore::GStreamerVideoDecoder::flush): (WebCore::GStreamerVideoDecoder::reset): (WebCore::GStreamerVideoDecoder::close): (WebCore::GStreamerInternalVideoDecoder::GStreamerInternalVideoDecoder): (WebCore::GStreamerInternalVideoDecoder::decode): (WebCore::GStreamerInternalVideoDecoder::flush): * Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.h: Added. * Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.cpp: Added. (WebCore::gstWorkQueue): (WebCore::GStreamerInternalVideoEncoder::create): (WebCore::GStreamerInternalVideoEncoder::postTask): (WebCore::GStreamerInternalVideoEncoder::close): (WebCore::GStreamerInternalVideoEncoder::harness const): (WebCore::GStreamerInternalVideoEncoder::isClosed const): (WebCore::GStreamerVideoEncoder::create): (WebCore::GStreamerVideoEncoder::GStreamerVideoEncoder): (WebCore::GStreamerVideoEncoder::~GStreamerVideoEncoder): (WebCore::GStreamerVideoEncoder::initialize): (WebCore::GStreamerVideoEncoder::encode): (WebCore::GStreamerVideoEncoder::flush): (WebCore::GStreamerVideoEncoder::reset): (WebCore::GStreamerVideoEncoder::close): (WebCore::GStreamerInternalVideoEncoder::GStreamerInternalVideoEncoder): (WebCore::GStreamerInternalVideoEncoder::initialize): (WebCore::GStreamerInternalVideoEncoder::encode): (WebCore::GStreamerInternalVideoEncoder::flush): * Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.h: Added. * Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp: (WebCore::GstSampleColorConverter::convertSample): (WebCore::GstSampleColorConverter::convertSampleToImage): (WebCore::VideoFrame::fromNativeImage): (WebCore::copyToGstBufferPlane): (WebCore::VideoFrame::createNV12): (WebCore::VideoFrame::createRGBA): (WebCore::VideoFrame::createBGRA): (WebCore::VideoFrame::createI420): (WebCore::VideoFrameGStreamer::create): (WebCore::VideoFrameGStreamer::createWrappedSample): (WebCore::VideoFrameGStreamer::VideoFrameGStreamer): (WebCore::copyPlane): (WebCore::VideoFrame::copyTo): (WebCore::VideoFrame::paintInContext): * Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.h: * Source/WebCore/platform/gstreamer/GStreamerElementHarness.cpp: Added. (WebCore::GStreamerElementHarness::GStreamerElementHarness): (WebCore::GStreamerElementHarness::~GStreamerElementHarness): (WebCore::GStreamerElementHarness::start): (WebCore::GStreamerElementHarness::pushStickyEvents): (WebCore::GStreamerElementHarness::pushSample): (WebCore::GStreamerElementHarness::pushBuffer): (WebCore::GStreamerElementHarness::pushEvent): (WebCore::GStreamerElementHarness::pullBuffer): (WebCore::GStreamerElementHarness::pullEvent): (WebCore::GStreamerElementHarness::outputCaps): (WebCore::GStreamerElementHarness::chainBuffer): (WebCore::GStreamerElementHarness::sinkQuery): (WebCore::GStreamerElementHarness::sinkEvent): (WebCore::GStreamerElementHarness::srcQuery): (WebCore::GStreamerElementHarness::srcEvent): (WebCore::GStreamerElementHarness::processOutputBuffers): (WebCore::GStreamerElementHarness::flush): * Source/WebCore/platform/gstreamer/GStreamerElementHarness.h: Added. (WebCore::GStreamerElementHarness::create): (WebCore::GStreamerElementHarness::element const): * Source/cmake/OptionsGTK.cmake: * Source/cmake/OptionsWPE.cmake: * Tools/TestWebKitAPI/PlatformGTK.cmake: * Tools/TestWebKitAPI/PlatformWPE.cmake: * Tools/TestWebKitAPI/Tests/WebCore/gstreamer/GstElementHarness.cpp: Added. (TestWebKitAPI::TEST_F): Canonical link: https://commits.webkit.org/258949@main
93ae0b1
to
c6f4630
Compare
Committed 258949@main (c6f4630): https://commits.webkit.org/258949@main Reviewed commits have been landed. Closing PR #8399 and removing active labels. |
c6f4630
19f922e
π§ͺ api-macπ§ͺ gtk-wk2