Skip to content

Commit

Permalink
WebRTC remote video codec factories should check video format strings…
Browse files Browse the repository at this point in the history
… in a case insensitive way

https://bugs.webkit.org/show_bug.cgi?id=261099
rdar://114839348

Reviewed by Jean-Yves Avenard.

Make WebRTC codec checks case insensitive, which follows what other browsers are doing.
Add a WebRTC test case to cover this case.

Keep doing case sensitive checks for WebCodecs and beef up WPT webcodecs tests accordingly.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-decoder.https.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-decoder.https.any.js:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-decoder.https.any.worker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-encoder-config.https.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-encoder-config.https.any.js:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/video-encoder-config.https.any.worker-expected.txt:
* LayoutTests/webrtc/video-lowercase-media-subtype-expected.txt: Added.
* LayoutTests/webrtc/video-lowercase-media-subtype.html: Added.
* LayoutTests/platform/glib/TestExpectations:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::createVideoDecoder):
(WebKit::createVideoEncoder):

Canonical link: https://commits.webkit.org/267637@main
  • Loading branch information
youennf committed Sep 5, 2023
1 parent fdabe49 commit 758b9fa
Show file tree
Hide file tree
Showing 10 changed files with 122 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Unrecogn
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Unrecognized codec with dataview description
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Audio codec
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Ambiguous codec
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Codec with bad casing
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Codec with MIME type
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Possible future H264 codec string
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Possible future HEVC codec string
Expand All @@ -16,6 +17,7 @@ PASS Test that VideoDecoder.configure() doesn't support config: Unrecognized cod
PASS Test that VideoDecoder.configure() doesn't support config: Unrecognized codec with dataview description
PASS Test that VideoDecoder.configure() doesn't support config: Audio codec
PASS Test that VideoDecoder.configure() doesn't support config: Ambiguous codec
PASS Test that VideoDecoder.configure() doesn't support config: Codec with bad casing
PASS Test that VideoDecoder.configure() doesn't support config: Codec with MIME type
PASS Test that VideoDecoder.configure() doesn't support config: Possible future H264 codec string
PASS Test that VideoDecoder.configure() doesn't support config: Possible future HEVC codec string
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ const validButUnsupportedConfigs = [
comment: 'Ambiguous codec',
config: {codec: 'vp9'},
},
{
comment: 'Codec with bad casing',
config: {codec: 'Vp09.00.10.08'},
},
{
comment: 'Codec with MIME type',
config: {codec: 'video/webm; codecs="vp8"'},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Unrecogn
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Unrecognized codec with dataview description
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Audio codec
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Ambiguous codec
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Codec with bad casing
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Codec with MIME type
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Possible future H264 codec string
PASS Test that VideoDecoder.isConfigSupported() doesn't support config: Possible future HEVC codec string
Expand All @@ -16,6 +17,7 @@ PASS Test that VideoDecoder.configure() doesn't support config: Unrecognized cod
PASS Test that VideoDecoder.configure() doesn't support config: Unrecognized codec with dataview description
PASS Test that VideoDecoder.configure() doesn't support config: Audio codec
PASS Test that VideoDecoder.configure() doesn't support config: Ambiguous codec
PASS Test that VideoDecoder.configure() doesn't support config: Codec with bad casing
PASS Test that VideoDecoder.configure() doesn't support config: Codec with MIME type
PASS Test that VideoDecoder.configure() doesn't support config: Possible future H264 codec string
PASS Test that VideoDecoder.configure() doesn't support config: Possible future HEVC codec string
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ PASS Test that VideoEncoder.configure() rejects invalid config: displayWidth is
PASS Test that VideoEncoder.configure() rejects invalid config: displayHeight is 0
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Invalid scalability mode
FAIL Test that VideoEncoder.isConfigSupported() doesn't support config: Unrecognized codec promise_test: Unhandled rejection with value: object "TypeError: Config is not valid"
FAIL Test that VideoEncoder.isConfigSupported() doesn't support config: Codec with bad casing promise_test: Unhandled rejection with value: object "TypeError: Config is not valid"
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Width is too large
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Height is too large
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Too strenuous accelerated encoding parameters
Expand All @@ -27,6 +28,9 @@ FAIL Test that VideoEncoder.configure() doesn't support config: Invalid scalabil
FAIL Test that VideoEncoder.configure() doesn't support config: Unrecognized codec assert_throws_dom: function "() => {
codec.configure(entry.config);
}" threw object "TypeError: Config is invalid" that is not a DOMException NotSupportedError: property "code" is equal to undefined, expected 9
FAIL Test that VideoEncoder.configure() doesn't support config: Codec with bad casing assert_throws_dom: function "() => {
codec.configure(entry.config);
}" threw object "TypeError: Config is invalid" that is not a DOMException NotSupportedError: property "code" is equal to undefined, expected 9
FAIL Test that VideoEncoder.configure() doesn't support config: Width is too large assert_throws_dom: function "() => {
codec.configure(entry.config);
}" did not throw
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,14 @@ const validButUnsupportedConfigs = [
height: 480,
},
},
{
comment: 'Codec with bad casing',
config: {
codec: 'vP8',
width: 640,
height: 480,
},
},
{
comment: 'Width is too large',
config: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ PASS Test that VideoEncoder.configure() rejects invalid config: displayWidth is
PASS Test that VideoEncoder.configure() rejects invalid config: displayHeight is 0
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Invalid scalability mode
FAIL Test that VideoEncoder.isConfigSupported() doesn't support config: Unrecognized codec promise_test: Unhandled rejection with value: object "TypeError: Config is not valid"
FAIL Test that VideoEncoder.isConfigSupported() doesn't support config: Codec with bad casing promise_test: Unhandled rejection with value: object "TypeError: Config is not valid"
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Width is too large
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Height is too large
PASS Test that VideoEncoder.isConfigSupported() doesn't support config: Too strenuous accelerated encoding parameters
Expand All @@ -27,6 +28,9 @@ FAIL Test that VideoEncoder.configure() doesn't support config: Invalid scalabil
FAIL Test that VideoEncoder.configure() doesn't support config: Unrecognized codec assert_throws_dom: function "() => {
codec.configure(entry.config);
}" threw object "TypeError: Config is invalid" that is not a DOMException NotSupportedError: property "code" is equal to undefined, expected 9
FAIL Test that VideoEncoder.configure() doesn't support config: Codec with bad casing assert_throws_dom: function "() => {
codec.configure(entry.config);
}" threw object "TypeError: Config is invalid" that is not a DOMException NotSupportedError: property "code" is equal to undefined, expected 9
FAIL Test that VideoEncoder.configure() doesn't support config: Width is too large assert_throws_dom: function "() => {
codec.configure(entry.config);
}" did not throw
Expand Down
2 changes: 2 additions & 0 deletions LayoutTests/platform/glib/TestExpectations
Original file line number Diff line number Diff line change
Expand Up @@ -1836,6 +1836,8 @@ webkit.org/b/256758 fast/mediastream/captureStream/canvas3d.html [ Failure ]

webkit.org/b/194611 http/wpt/webrtc/getUserMedia-processSwapping.html [ Failure ]

webkit.org/b/261099 webrtc/video-lowercase-media-subtype.html [ Skip ]

# DataChannel GstWebRTC implementation incomplete, missing stats support.
webkit.org/b/235885 webrtc/datachannel/datachannel-stats.html [ Skip ]
webkit.org/b/235885 webrtc/datachannel/getStats-no-prflx-remote-candidate.html [ Skip ]
Expand Down
5 changes: 5 additions & 0 deletions LayoutTests/webrtc/video-lowercase-media-subtype-expected.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@


PASS Video exchange with lower case h264 on encoding side
PASS Video exchange with lower case h264 on decoding side

83 changes: 83 additions & 0 deletions LayoutTests/webrtc/video-lowercase-media-subtype.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>Lowercase media subtype name in SDP</title>
<script src="../resources/testharness.js"></script>
<script src="../resources/testharnessreport.js"></script>
</head>
<body>
<video id="video1" playsinline autoplay=""></video>
<video id="video2" playsinline autoplay=""></video>
<script>
promise_test(async (test) => {
const localStream = await navigator.mediaDevices.getUserMedia({video:true });

const pc1 = new RTCPeerConnection();
const pc2 = new RTCPeerConnection();

pc2.addTrack(localStream.getVideoTracks()[0], localStream);
pc1.addTransceiver("video", {direction:"recvonly"});

const codecs = RTCRtpSender.getCapabilities("video").codecs;
const h264Codec = codecs.filter(codec => { return codec.mimeType === "video/H264"; })[0];
pc1.getTransceivers().forEach((transceiver) => { transceiver.setCodecPreferences([h264Codec]); });

pc1.onicecandidate = (e) => pc2.addIceCandidate(e.candidate);
pc2.onicecandidate = (e) => pc1.addIceCandidate(e.candidate);

await pc1.setLocalDescription();
let desc = pc1.localDescription;

const regex = /H264/i;
desc.sdp = desc.sdp.replace(regex, "h264");
await pc2.setRemoteDescription(desc);

const answer = await pc2.createAnswer();
await pc1.setRemoteDescription(answer);

answer.sdp = answer.sdp.replace(regex, "h264");
await pc2.setLocalDescription(answer);

video1.srcObject = new MediaStream([pc1.getReceivers()[0].track]);
await video1.play();
}, "Video exchange with lower case h264 on encoding side");

promise_test(async (test) => {
const localStream = await navigator.mediaDevices.getUserMedia({video:true });

const pc1 = new RTCPeerConnection();
const pc2 = new RTCPeerConnection();

pc1.addTrack(localStream.getVideoTracks()[0], localStream);
const remoteStreamPromise = new Promise((resolve, reject) => {
pc2.ontrack = (e) => resolve(e.streams[0]);
setTimeout(() => reject("no remote track"), 5000);
});

const codecs = RTCRtpSender.getCapabilities("video").codecs;
const h264Codec = codecs.filter(codec => { return codec.mimeType === "video/H264"; })[0];
pc1.getTransceivers().forEach((transceiver) => { transceiver.setCodecPreferences([h264Codec]); });

pc1.onicecandidate = (e) => pc2.addIceCandidate(e.candidate);
pc2.onicecandidate = (e) => pc1.addIceCandidate(e.candidate);

await pc1.setLocalDescription();
let desc = pc1.localDescription;

const regex = /H264/i;
desc.sdp = desc.sdp.replace(regex, "h264");
await pc2.setRemoteDescription(desc);

const answer = await pc2.createAnswer();
await pc1.setRemoteDescription(answer);

answer.sdp = answer.sdp.replace(regex, "h264");
await pc2.setLocalDescription(answer);

video2.srcObject = await remoteStreamPromise;
await video2.play();
}, "Video exchange with lower case h264 on decoding side");
</script>
</body>
</html>
14 changes: 8 additions & 6 deletions Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -65,16 +65,18 @@ using namespace WebCore;
static webrtc::WebKitVideoDecoder createVideoDecoder(const webrtc::SdpVideoFormat& format)
{
auto& codecs = WebProcess::singleton().libWebRTCCodecs();
if (format.name == "H264")
auto codecString = String::fromUTF8(format.name.data(), format.name.length());

if (equalIgnoringASCIICase(codecString, "H264"_s))
return { codecs.createDecoder(VideoCodecType::H264), false };

if (format.name == "H265")
if (equalIgnoringASCIICase(codecString, "H265"_s))
return { codecs.createDecoder(VideoCodecType::H265), false };

if (format.name == "VP9" && codecs.supportVP9VTB())
if (equalIgnoringASCIICase(codecString, "VP9"_s) && codecs.supportVP9VTB())
return { codecs.createDecoder(VideoCodecType::VP9), false };

if (format.name == "AV1") {
if (equalIgnoringASCIICase(codecString, "AV1"_s)) {
auto av1Decoder = createLibWebRTCDav1dDecoder().moveToUniquePtr();
return { av1Decoder.release(), true };
}
Expand Down Expand Up @@ -124,10 +126,10 @@ static int32_t registerDecodeCompleteCallback(webrtc::WebKitVideoDecoder::Value

static webrtc::WebKitVideoEncoder createVideoEncoder(const webrtc::SdpVideoFormat& format)
{
if (format.name == "H264")
if (format.name == "H264" || format.name == "h264")
return WebProcess::singleton().libWebRTCCodecs().createEncoder(VideoCodecType::H264, format.parameters);

if (format.name == "H265")
if (format.name == "H265" || format.name == "h265")
return WebProcess::singleton().libWebRTCCodecs().createEncoder(VideoCodecType::H265, format.parameters);

return nullptr;
Expand Down

0 comments on commit 758b9fa

Please sign in to comment.