Skip to content
Permalink
Browse files
Implement WebCodecsVideoDecoder with VPx backend
https://bugs.webkit.org/show_bug.cgi?id=245878
rdar://problem/100609725

Introduce VideoDecoder interface that is used by WebCodecsVideoDecoder as the interface to actual decoders.
Implement WebCodecs VideoDecoder algorithms.
Add VP8 and VP9 decoder implementations for VideoDecoder.

Reviewed by Eric Carlson.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-decoder.https.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-decoder.https.any.worker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoDecoder-codec-specific.https.any.worker_vp8-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoDecoder-codec-specific.https.any_vp8-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoFrame-canvasImageSource-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoFrame-construction.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoFrame-construction.window-expected.txt:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/WebCore/Modules/webcodecs/WebCodecsEncodedVideoChunk.h:
(WebCore::WebCodecsEncodedVideoChunk::byteLength const):
(WebCore::WebCodecsEncodedVideoChunk::data const):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoder.cpp:
(WebCore::WebCodecsVideoDecoder::configure):
(WebCore::WebCodecsVideoDecoder::decode):
(WebCore::WebCodecsVideoDecoder::flush):
(WebCore::WebCodecsVideoDecoder::reset):
(WebCore::WebCodecsVideoDecoder::close):
(WebCore::isValidDecoderConfig):
(WebCore::WebCodecsVideoDecoder::isConfigSupported):
(WebCore::WebCodecsVideoDecoder::closeDecoder):
(WebCore::WebCodecsVideoDecoder::resetDecoder):
(WebCore::WebCodecsVideoDecoder::scheduleDequeueEvent):
(WebCore::WebCodecsVideoDecoder::setInternalDecoder):
(WebCore::WebCodecsVideoDecoder::queueControlMessageAndProcess):
(WebCore::WebCodecsVideoDecoder::processControlMessageQueue):
(WebCore::WebCore::WebCodecsVideoDecoder::suspend):
(WebCore::WebCodecsVideoDecoder::stop):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoder.h:
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoderConfig.h:
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoderConfig.idl:
* Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoderSupport.idl:
* Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp:
(WebCore::WebCodecsVideoFrame::create):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.h:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/LibWebRTCVPXVideoDecoder.cpp: Added.
(WebCore::vpxQueue):
(WebCore::LibWebRTCVPXInternalVideoDecoder::create):
(WebCore::LibWebRTCVPXInternalVideoDecoder::postTask):
(WebCore::LibWebRTCVPXInternalVideoDecoder::close):
(WebCore::LibWebRTCVPXVideoDecoder::LibWebRTCVPXVideoDecoder):
(WebCore::LibWebRTCVPXVideoDecoder::~LibWebRTCVPXVideoDecoder):
(WebCore::LibWebRTCVPXVideoDecoder::decode):
(WebCore::LibWebRTCVPXVideoDecoder::flush):
(WebCore::LibWebRTCVPXVideoDecoder::reset):
(WebCore::LibWebRTCVPXVideoDecoder::close):
(WebCore::LibWebRTCVPXInternalVideoDecoder::decode):
(WebCore::createInternalDecoder):
(WebCore::LibWebRTCVPXInternalVideoDecoder::LibWebRTCVPXInternalVideoDecoder):
(WebCore::LibWebRTCVPXInternalVideoDecoder::Decoded):
* Source/WebCore/platform/LibWebRTCVPXVideoDecoder.h: Added.
* Source/WebCore/platform/VideoDecoder.cpp: Added.
(WebCore::VideoDecoder::create):
* Source/WebCore/platform/VideoDecoder.h: Added.
* Source/WebCore/platform/mediastream/libwebrtc/LibWebRTCProviderCocoa.cpp:

Canonical link: https://commits.webkit.org/255138@main
  • Loading branch information
youennf committed Oct 4, 2022
1 parent 70e23de commit db7df81e7dc760e7dbabec0093f07d1549330dd0
Show file tree
Hide file tree
Showing 24 changed files with 711 additions and 135 deletions.
@@ -1,19 +1,9 @@

FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Empty codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Unrecognized codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Audio codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Ambiguous codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Codec with MIME type promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Empty codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Unrecognized codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Audio codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Ambiguous codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Codec with MIME type
FAIL Test that VideoDecoder.configure() rejects invalid config:Empty codec assert_throws_js: function "() => {
codec.configure(entry.config);
}" did not throw
@@ -1,19 +1,9 @@

FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Empty codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Unrecognized codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Audio codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Ambiguous codec promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
FAIL Test that VideoDecoder.isConfigSupported() rejects invalid config:Codec with MIME type promise_rejects_js: function "function () { throw e }" threw object "NotSupportedError: Not implemented" ("NotSupportedError") expected instance of function "function TypeError() {
[native code]
}" ("TypeError")
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Empty codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Unrecognized codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Audio codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Ambiguous codec
PASS Test that VideoDecoder.isConfigSupported() rejects invalid config:Codec with MIME type
FAIL Test that VideoDecoder.configure() rejects invalid config:Empty codec assert_throws_js: function "() => {
codec.configure(entry.config);
}" did not throw
@@ -1,22 +1,20 @@

Harness Error (FAIL), message = Error: vp8 unsupported

NOTRUN Test isConfigSupported()
NOTRUN Test isConfigSupported() with 1080p crop
NOTRUN Test that isConfigSupported() returns a parsed configuration
NOTRUN Test invalid configs
NOTRUN Test configure()
NOTRUN Decode a key frame
NOTRUN Decode a non key frame first fails
NOTRUN Verify reset() suppresses outputs
NOTRUN Test unconfigured VideoDecoder operations
NOTRUN Test closed VideoDecoder operations
NOTRUN Decode empty frame
NOTRUN Decode corrupt frame
NOTRUN Close while decoding corrupt frame
NOTRUN Test decoding after flush
NOTRUN Test decoding a with negative timestamp
NOTRUN Test reset during flush
NOTRUN Test low-latency decoding
NOTRUN VideoDecoder decodeQueueSize test
PASS Test isConfigSupported()
PASS Test isConfigSupported() with 1080p crop
PASS Test that isConfigSupported() returns a parsed configuration
FAIL Test invalid configs assert_unreached: Should have rejected: invalid codedWidth Reached unreachable code
PASS Test configure()
FAIL Decode a key frame assert_equals: duration expected 1 but got 0
PASS Decode a non key frame first fails
FAIL Verify reset() suppresses outputs assert_greater_than: expected a number greater than 0 but got 0
PASS Test unconfigured VideoDecoder operations
PASS Test closed VideoDecoder operations
FAIL Decode empty frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error 1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
FAIL Decode corrupt frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error -1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
PASS Close while decoding corrupt frame
PASS Test decoding after flush
FAIL Test decoding a with negative timestamp assert_equals: timestamp expected -42 but got 0
PASS Test reset during flush
PASS Test low-latency decoding
PASS VideoDecoder decodeQueueSize test

@@ -1,22 +1,20 @@

Harness Error (FAIL), message = Error: vp09.00.10.08 unsupported

NOTRUN Test isConfigSupported()
NOTRUN Test isConfigSupported() with 1080p crop
NOTRUN Test that isConfigSupported() returns a parsed configuration
NOTRUN Test invalid configs
NOTRUN Test configure()
NOTRUN Decode a key frame
NOTRUN Decode a non key frame first fails
NOTRUN Verify reset() suppresses outputs
NOTRUN Test unconfigured VideoDecoder operations
NOTRUN Test closed VideoDecoder operations
NOTRUN Decode empty frame
NOTRUN Decode corrupt frame
NOTRUN Close while decoding corrupt frame
NOTRUN Test decoding after flush
NOTRUN Test decoding a with negative timestamp
NOTRUN Test reset during flush
NOTRUN Test low-latency decoding
NOTRUN VideoDecoder decodeQueueSize test
PASS Test isConfigSupported()
PASS Test isConfigSupported() with 1080p crop
PASS Test that isConfigSupported() returns a parsed configuration
FAIL Test invalid configs assert_unreached: Should have rejected: invalid codedWidth Reached unreachable code
PASS Test configure()
FAIL Decode a key frame assert_equals: duration expected 1 but got 0
PASS Decode a non key frame first fails
FAIL Verify reset() suppresses outputs assert_greater_than: expected a number greater than 0 but got 0
PASS Test unconfigured VideoDecoder operations
PASS Test closed VideoDecoder operations
FAIL Decode empty frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error 1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
FAIL Decode corrupt frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error -1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
PASS Close while decoding corrupt frame
PASS Test decoding after flush
FAIL Test decoding a with negative timestamp assert_equals: timestamp expected -42 but got 0
PASS Test reset during flush
PASS Test low-latency decoding
PASS VideoDecoder decodeQueueSize test

@@ -1,22 +1,20 @@

Harness Error (FAIL), message = Error: vp8 unsupported

NOTRUN Test isConfigSupported()
NOTRUN Test isConfigSupported() with 1080p crop
NOTRUN Test that isConfigSupported() returns a parsed configuration
NOTRUN Test invalid configs
NOTRUN Test configure()
NOTRUN Decode a key frame
NOTRUN Decode a non key frame first fails
NOTRUN Verify reset() suppresses outputs
NOTRUN Test unconfigured VideoDecoder operations
NOTRUN Test closed VideoDecoder operations
NOTRUN Decode empty frame
NOTRUN Decode corrupt frame
NOTRUN Close while decoding corrupt frame
NOTRUN Test decoding after flush
NOTRUN Test decoding a with negative timestamp
NOTRUN Test reset during flush
NOTRUN Test low-latency decoding
NOTRUN VideoDecoder decodeQueueSize test
PASS Test isConfigSupported()
PASS Test isConfigSupported() with 1080p crop
PASS Test that isConfigSupported() returns a parsed configuration
FAIL Test invalid configs assert_unreached: Should have rejected: invalid codedWidth Reached unreachable code
PASS Test configure()
FAIL Decode a key frame assert_equals: duration expected 1 but got 0
PASS Decode a non key frame first fails
FAIL Verify reset() suppresses outputs assert_greater_than: expected a number greater than 0 but got 0
PASS Test unconfigured VideoDecoder operations
PASS Test closed VideoDecoder operations
FAIL Decode empty frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error 1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
FAIL Decode corrupt frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error -1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
PASS Close while decoding corrupt frame
PASS Test decoding after flush
FAIL Test decoding a with negative timestamp assert_equals: timestamp expected -42 but got 0
PASS Test reset during flush
PASS Test low-latency decoding
PASS VideoDecoder decodeQueueSize test

@@ -1,22 +1,20 @@

Harness Error (FAIL), message = Error: vp09.00.10.08 unsupported

NOTRUN Test isConfigSupported()
NOTRUN Test isConfigSupported() with 1080p crop
NOTRUN Test that isConfigSupported() returns a parsed configuration
NOTRUN Test invalid configs
NOTRUN Test configure()
NOTRUN Decode a key frame
NOTRUN Decode a non key frame first fails
NOTRUN Verify reset() suppresses outputs
NOTRUN Test unconfigured VideoDecoder operations
NOTRUN Test closed VideoDecoder operations
NOTRUN Decode empty frame
NOTRUN Decode corrupt frame
NOTRUN Close while decoding corrupt frame
NOTRUN Test decoding after flush
NOTRUN Test decoding a with negative timestamp
NOTRUN Test reset during flush
NOTRUN Test low-latency decoding
NOTRUN VideoDecoder decodeQueueSize test
PASS Test isConfigSupported()
PASS Test isConfigSupported() with 1080p crop
PASS Test that isConfigSupported() returns a parsed configuration
FAIL Test invalid configs assert_unreached: Should have rejected: invalid codedWidth Reached unreachable code
PASS Test configure()
FAIL Decode a key frame assert_equals: duration expected 1 but got 0
PASS Decode a non key frame first fails
FAIL Verify reset() suppresses outputs assert_greater_than: expected a number greater than 0 but got 0
PASS Test unconfigured VideoDecoder operations
PASS Test closed VideoDecoder operations
FAIL Decode empty frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error 1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
FAIL Decode corrupt frame promise_rejects_dom: function "function () { throw e }" threw object "EncodingError: VPx decoding failed with error -1" that is not a DOMException AbortError: property "code" is equal to 0, expected 20
PASS Close while decoding corrupt frame
PASS Test decoding after flush
FAIL Test decoding a with negative timestamp assert_equals: timestamp expected -42 but got 0
PASS Test reset during flush
PASS Test low-latency decoding
PASS VideoDecoder decodeQueueSize test

@@ -336,3 +336,12 @@ __ZTVN3rtc14NetworkManagerE
__ZN3rtc18NetworkManagerBaseC2EPKN6webrtc15FieldTrialsViewE
__ZN3rtc13SocketAddressC1EN4absl11string_viewEi
__ZN3rtc17AsyncPacketSocket14SubscribeCloseEPKvNSt3__18functionIFvPS0_iEEE
__ZN6webrtc10VP8Decoder6CreateEv
__ZN6webrtc10VP9Decoder6CreateEv
__ZN6webrtc12EncodedImageC1Ev
__ZN6webrtc12EncodedImageD1Ev
__ZN6webrtc12VideoDecoder8Settings19set_number_of_coresEi
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEN4absl8optionalIiEENS4_IhEE
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEx
__ZN6webrtc7CpuInfo19DetectNumberOfCoresEv
__ZTVN6webrtc20DecodedImageCallbackE
@@ -336,3 +336,12 @@ __ZTVN3rtc14NetworkManagerE
__ZN3rtc18NetworkManagerBaseC2EPKN6webrtc15FieldTrialsViewE
__ZN3rtc13SocketAddressC1EN4absl11string_viewEi
__ZN3rtc17AsyncPacketSocket14SubscribeCloseEPKvNSt3__18functionIFvPS0_iEEE
__ZN6webrtc10VP8Decoder6CreateEv
__ZN6webrtc10VP9Decoder6CreateEv
__ZN6webrtc12EncodedImageC1Ev
__ZN6webrtc12EncodedImageD1Ev
__ZN6webrtc12VideoDecoder8Settings19set_number_of_coresEi
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEN4absl8optionalIiEENS4_IhEE
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEx
__ZN6webrtc7CpuInfo19DetectNumberOfCoresEv
__ZTVN6webrtc20DecodedImageCallbackE
@@ -336,3 +336,12 @@ __ZTVN3rtc14NetworkManagerE
__ZN3rtc18NetworkManagerBaseC2EPKN6webrtc15FieldTrialsViewE
__ZN3rtc13SocketAddressC1EN4absl11string_viewEi
__ZN3rtc17AsyncPacketSocket14SubscribeCloseEPKvNSt3__18functionIFvPS0_iEEE
__ZN6webrtc10VP8Decoder6CreateEv
__ZN6webrtc10VP9Decoder6CreateEv
__ZN6webrtc12EncodedImageC1Ev
__ZN6webrtc12EncodedImageD1Ev
__ZN6webrtc12VideoDecoder8Settings19set_number_of_coresEi
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEN4absl8optionalIiEENS4_IhEE
__ZN6webrtc20DecodedImageCallback7DecodedERNS_10VideoFrameEx
__ZN6webrtc7CpuInfo19DetectNumberOfCoresEv
__ZTVN6webrtc20DecodedImageCallbackE
@@ -50,10 +50,12 @@ class WebCodecsEncodedVideoChunk : public RefCounted<WebCodecsEncodedVideoChunk>
WebCodecsEncodedVideoChunkType type() const { return m_type; };
int64_t timestamp() const { return m_timestamp; }
std::optional<uint64_t> duration() const { return m_duration; }
size_t byteLength() const { return m_data.capacity(); }
size_t byteLength() const { return m_data.size(); }

ExceptionOr<void> copyTo(BufferSource&&);

const uint8_t* data() const { return m_data.data(); }

private:
explicit WebCodecsEncodedVideoChunk(Init&&);

0 comments on commit db7df81

Please sign in to comment.