Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for AVC H264 WebCodecsVideoDecoder #5074

Conversation

youennf
Copy link
Contributor

@youennf youennf commented Oct 6, 2022

a248b49

Add support for AVC H264 WebCodecsVideoDecoder
https://bugs.webkit.org/show_bug.cgi?id=246141
rdar://problem/100844377

Reviewed by Eric Carlson.

Update RTCVideoDecoderH264 and RTCVideoDecoderH265 to support AVC.
To do so, we introduce setAVCFormat to setup the decoder and put it in AVC mode.
When receiving data to decode, we skip the AnnexB translation since VTB is AVC based.

Update WebKit layer to pass the AVC description data to GPUProcess.

Covered by layout tests.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoDecoder-codec-specific.https.any.worker_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/videoDecoder-codec-specific.https.any_h264_avc-expected.txt:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm:
(-[WK_RTCLocalVideoH264H265VP9Decoder setFormat:size:width:height:]):
(webrtc::setDecodingFormat):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm:
(-[RTCVideoDecoderH264 init]):
(H264BufferToCMSampleBuffer):
(-[RTCVideoDecoderH264 decodeData:size:timeStamp:]):
(-[RTCVideoDecoderH264 setAVCFormat:size:width:height:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm:
(-[RTCVideoDecoderH265 init]):
(H265BufferToCMSampleBuffer):
(-[RTCVideoDecoderH265 decodeData:size:timeStamp:]):
(-[RTCVideoDecoderH265 setAVCFormat:size:width:height:]):
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::setDecoderFormatDescription):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoCodecFactory::createDecoder):
(WebKit::RemoteVideoDecoder::RemoteVideoDecoder):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::setDecoderFormatDescription):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:

Canonical link: https://commits.webkit.org/255422@main

a7371e3

Misc iOS, tvOS & watchOS macOS Linux Windows
βœ… πŸ§ͺ style βœ… πŸ›  ios βœ… πŸ›  mac βœ… πŸ›  wpe   πŸ›  πŸ§ͺ win
βœ… πŸ›  ios-sim βœ… πŸ›  mac-debug βœ… πŸ›  gtk   πŸ›  wincairo
βœ… πŸ§ͺ webkitperl   πŸ§ͺ ios-wk2 βœ… πŸ›  mac-AS-debug βœ… πŸ§ͺ gtk-wk2
  πŸ§ͺ api-ios   πŸ§ͺ api-mac   πŸ§ͺ api-gtk
βœ… πŸ›  tv   πŸ§ͺ mac-wk1
βœ… πŸ›  tv-sim   πŸ§ͺ mac-wk2
βœ… πŸ›  πŸ§ͺ merge βœ… πŸ›  watch   πŸ§ͺ mac-AS-debug-wk2
βœ… πŸ›  watch-sim βœ… πŸ§ͺ mac-wk2-stress

@youennf youennf requested a review from cdumez as a code owner October 6, 2022 09:40
@youennf youennf self-assigned this Oct 6, 2022
@youennf youennf added Media Bugs related to the HTML 5 Media elements. WebKit Nightly Build labels Oct 6, 2022
@youennf youennf marked this pull request as draft October 6, 2022 09:41
@youennf youennf force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoDecoder branch from b526c13 to a794200 Compare October 6, 2022 12:56
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Oct 6, 2022
@youennf youennf force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoDecoder branch from a794200 to 6ff0daf Compare October 10, 2022 09:35
@youennf youennf marked this pull request as ready for review October 10, 2022 10:29
Copy link
Contributor

@eric-carlson eric-carlson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me

@youennf youennf force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoDecoder branch from 6ff0daf to a7371e3 Compare October 12, 2022 06:47
@youennf youennf added merge-queue Applied to send a pull request to merge-queue and removed merging-blocked Applied to prevent a change from being merged labels Oct 12, 2022
@webkit-commit-queue
Copy link
Collaborator

Committed 255422@main (a248b49): https://commits.webkit.org/255422@main

Reviewed commits have been landed. Closing PR #5074 and removing active labels.

@webkit-early-warning-system webkit-early-warning-system force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoDecoder branch from a7371e3 to a248b49 Compare October 12, 2022 08:41
@webkit-commit-queue webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Media Bugs related to the HTML 5 Media elements.
Projects
None yet
5 participants