Skip to content

Add support for AVC H264 WebCodecsVideoEncoder #5192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

youennf
Copy link
Contributor

@youennf youennf commented Oct 10, 2022

a591b26

Add support for AVC H264 WebCodecsVideoEncoder
https://bugs.webkit.org/show_bug.cgi?id=246150
rdar://problem/100851081

Reviewed by Eric Carlson.

Add support for AVC encoder in libwebrtc.
Add support at WebKit layer by passing a boolean to set annexB or not, and a callback to get the description in case of AVC.
Add support at WebCore level to provide the metadata in the output callback when needed.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/full-cycle-test.https.any_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any.worker_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any_h264_avc-expected.txt:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(-[WK_RTCLocalVideoH264H265Encoder setUseAnnexB:]):
(-[WK_RTCLocalVideoH264H265Encoder setDescriptionCallback:]):
(webrtc::createLocalEncoder):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 initWithCodecInfo:]):
(-[RTCVideoEncoderH264 setUseAnnexB:]):
(-[RTCVideoEncoderH264 setDescriptionCallback:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:rotation:isKeyFrameRequired:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm:
(-[RTCVideoEncoderH265 initWithCodecInfo:]):
(-[RTCVideoEncoderH265 setUseAnnexB:]):
(-[RTCVideoEncoderH265 setDescriptionCallback:]):
(-[RTCVideoEncoderH265 frameWasEncoded:flags:sampleBuffer:width:height:renderTimeMs:timestamp:rotation:]):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoder.cpp:
(WebCore::WebCodecsVideoEncoder::configure):
(WebCore::WebCodecsVideoEncoder::createEncodedChunkMetadata):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoder.h:
* Source/WebCore/platform/LibWebRTCVPXVideoEncoder.cpp:
(WebCore::LibWebRTCVPXVideoEncoder::LibWebRTCVPXVideoEncoder):
* Source/WebCore/platform/LibWebRTCVPXVideoEncoder.h:
* Source/WebCore/platform/VideoEncoder.cpp:
(WebCore::VideoEncoder::create):
(WebCore::VideoEncoder::createLocalEncoder):
* Source/WebCore/platform/VideoEncoder.h:
* Source/WebKit/Configurations/WebKit.xcconfig:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::createEncoder):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoEncoderCallbacks::create):
(WebKit::RemoteVideoCodecFactory::createEncoder):
(WebKit::RemoteVideoDecoder::RemoteVideoDecoder): Deleted.
(WebKit::RemoteVideoDecoder::~RemoteVideoDecoder): Deleted.
(WebKit::RemoteVideoDecoder::decode): Deleted.
(WebKit::RemoteVideoDecoder::flush): Deleted.
(WebKit::RemoteVideoDecoder::reset): Deleted.
(WebKit::RemoteVideoDecoder::close): Deleted.
(WebKit::RemoteVideoDecoderCallbacks::RemoteVideoDecoderCallbacks): Deleted.
(WebKit::RemoteVideoDecoderCallbacks::notifyVideoFrame): Deleted.
(WebKit::RemoteVideoEncoder::RemoteVideoEncoder): Deleted.
(WebKit::RemoteVideoEncoder::~RemoteVideoEncoder): Deleted.
(WebKit::RemoteVideoEncoder::encode): Deleted.
(WebKit::RemoteVideoEncoder::flush): Deleted.
(WebKit::RemoteVideoEncoder::reset): Deleted.
(WebKit::RemoteVideoEncoder::close): Deleted.
(WebKit::RemoteVideoEncoderCallbacks::RemoteVideoEncoderCallbacks): Deleted.
(WebKit::RemoteVideoEncoderCallbacks::notifyEncodedChunk): Deleted.
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::createEncoder):
(WebKit::LibWebRTCCodecs::createEncoderAndWaitUntilReady):
(WebKit::LibWebRTCCodecs::createEncoderInternal):
(WebKit::LibWebRTCCodecs::registerEncoderDescriptionCallback):
(WebKit::LibWebRTCCodecs::setEncodingDescription):
(WebKit::LibWebRTCCodecs::gpuProcessConnectionDidClose):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in:

Canonical link: https://commits.webkit.org/255430@main

cd244ef

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe 🛠 🧪 win
✅ 🧪 bindings ✅ 🛠 ios-sim ✅ 🛠 mac-debug ✅ 🛠 gtk 🛠 wincairo
✅ 🧪 webkitperl ✅ 🧪 ios-wk2 ✅ 🛠 mac-AS-debug ✅ 🧪 gtk-wk2
🧪 api-ios 🧪 api-mac 🧪 api-gtk
✅ 🛠 tv 🧪 mac-wk1
✅ 🛠 tv-sim 🧪 mac-wk2
✅ 🛠 🧪 merge ✅ 🛠 watch 🧪 mac-AS-debug-wk2
✅ 🛠 watch-sim ✅ 🧪 mac-wk2-stress

@youennf youennf self-assigned this Oct 10, 2022
@youennf youennf added Media Bugs related to the HTML 5 Media elements. WebKit Nightly Build labels Oct 10, 2022
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Oct 10, 2022
Copy link
Contributor

@eric-carlson eric-carlson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me

@youennf youennf marked this pull request as ready for review October 12, 2022 09:57
@youennf youennf requested a review from cdumez as a code owner October 12, 2022 09:57
@youennf youennf force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoEncoder branch from 0084e27 to 86f2915 Compare October 12, 2022 10:35
@youennf youennf force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoEncoder branch from 86f2915 to cd244ef Compare October 12, 2022 11:08
@youennf youennf added merge-queue Applied to send a pull request to merge-queue and removed merging-blocked Applied to prevent a change from being merged labels Oct 12, 2022
https://bugs.webkit.org/show_bug.cgi?id=246150
rdar://problem/100851081

Reviewed by Eric Carlson.

Add support for AVC encoder in libwebrtc.
Add support at WebKit layer by passing a boolean to set annexB or not, and a callback to get the description in case of AVC.
Add support at WebCore level to provide the metadata in the output callback when needed.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/full-cycle-test.https.any_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any.worker_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any_h264_avc-expected.txt:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(-[WK_RTCLocalVideoH264H265Encoder setUseAnnexB:]):
(-[WK_RTCLocalVideoH264H265Encoder setDescriptionCallback:]):
(webrtc::createLocalEncoder):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 initWithCodecInfo:]):
(-[RTCVideoEncoderH264 setUseAnnexB:]):
(-[RTCVideoEncoderH264 setDescriptionCallback:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:rotation:isKeyFrameRequired:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm:
(-[RTCVideoEncoderH265 initWithCodecInfo:]):
(-[RTCVideoEncoderH265 setUseAnnexB:]):
(-[RTCVideoEncoderH265 setDescriptionCallback:]):
(-[RTCVideoEncoderH265 frameWasEncoded:flags:sampleBuffer:width:height:renderTimeMs:timestamp:rotation:]):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoder.cpp:
(WebCore::WebCodecsVideoEncoder::configure):
(WebCore::WebCodecsVideoEncoder::createEncodedChunkMetadata):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoder.h:
* Source/WebCore/platform/LibWebRTCVPXVideoEncoder.cpp:
(WebCore::LibWebRTCVPXVideoEncoder::LibWebRTCVPXVideoEncoder):
* Source/WebCore/platform/LibWebRTCVPXVideoEncoder.h:
* Source/WebCore/platform/VideoEncoder.cpp:
(WebCore::VideoEncoder::create):
(WebCore::VideoEncoder::createLocalEncoder):
* Source/WebCore/platform/VideoEncoder.h:
* Source/WebKit/Configurations/WebKit.xcconfig:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::createEncoder):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoEncoderCallbacks::create):
(WebKit::RemoteVideoCodecFactory::createEncoder):
(WebKit::RemoteVideoDecoder::RemoteVideoDecoder): Deleted.
(WebKit::RemoteVideoDecoder::~RemoteVideoDecoder): Deleted.
(WebKit::RemoteVideoDecoder::decode): Deleted.
(WebKit::RemoteVideoDecoder::flush): Deleted.
(WebKit::RemoteVideoDecoder::reset): Deleted.
(WebKit::RemoteVideoDecoder::close): Deleted.
(WebKit::RemoteVideoDecoderCallbacks::RemoteVideoDecoderCallbacks): Deleted.
(WebKit::RemoteVideoDecoderCallbacks::notifyVideoFrame): Deleted.
(WebKit::RemoteVideoEncoder::RemoteVideoEncoder): Deleted.
(WebKit::RemoteVideoEncoder::~RemoteVideoEncoder): Deleted.
(WebKit::RemoteVideoEncoder::encode): Deleted.
(WebKit::RemoteVideoEncoder::flush): Deleted.
(WebKit::RemoteVideoEncoder::reset): Deleted.
(WebKit::RemoteVideoEncoder::close): Deleted.
(WebKit::RemoteVideoEncoderCallbacks::RemoteVideoEncoderCallbacks): Deleted.
(WebKit::RemoteVideoEncoderCallbacks::notifyEncodedChunk): Deleted.
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::createEncoder):
(WebKit::LibWebRTCCodecs::createEncoderAndWaitUntilReady):
(WebKit::LibWebRTCCodecs::createEncoderInternal):
(WebKit::LibWebRTCCodecs::registerEncoderDescriptionCallback):
(WebKit::LibWebRTCCodecs::setEncodingDescription):
(WebKit::LibWebRTCCodecs::gpuProcessConnectionDidClose):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in:

Canonical link: https://commits.webkit.org/255430@main
@webkit-commit-queue webkit-commit-queue force-pushed the eng/Add-support-for-AVC-H264-WebCodecsVideoEncoder branch from cd244ef to a591b26 Compare October 12, 2022 13:49
@webkit-commit-queue
Copy link
Collaborator

Committed 255430@main (a591b26): https://commits.webkit.org/255430@main

Reviewed commits have been landed. Closing PR #5192 and removing active labels.

@webkit-commit-queue webkit-commit-queue merged commit a591b26 into WebKit:main Oct 12, 2022
@webkit-commit-queue webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Media Bugs related to the HTML 5 Media elements.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants