Skip to content
Permalink
Browse files
[Mac] MSE-based video pauses when put in background; PiP context menu…
… disabled

https://bugs.webkit.org/show_bug.cgi?id=225958
<rdar://78130303>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/media-source/media-source-has-audio-video.html

In r277116, in order to reduce the CPU usage incurred when pushing state changes from
the GPU process to the WebContent process, state changes to `hasAudio` and `hasVideo`
were now only updated when the MediaPlayer indicated that its `characteristicsChanged()`.
However, in SourceBufferPrivateAVFObjC, this call was performed too early, before the
SourceBuffer in the WebContent process had informed SourceBufferPrivate of newly added
audio and video tracks. Thus the cached values of `hasAudio` and `hasVideo` inside the
WebContent process were never updated.

Call `characteristicsChanged()` from the callback handler of `didReceiveInitializationSegment()`,
which will be called only after the WebContent process completes its handling of the
new initialization segment, which includes telling the SourceBufferPrivate in the GPU
process about newly added audio and video tracks.

* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData):

LayoutTests:

* media/media-source/media-source-has-audio-video-expected.txt: Added.
* media/media-source/media-source-has-audio-video.html: Added.


Canonical link: https://commits.webkit.org/237903@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@277726 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
jernoble committed May 19, 2021
1 parent a10ec1d commit 4c7f38aeea917b816b89dd20331709d07167b203
Showing 5 changed files with 94 additions and 0 deletions.
@@ -1,3 +1,14 @@
2021-05-19 Jer Noble <jer.noble@apple.com>

[Mac] MSE-based video pauses when put in background; PiP context menu disabled
https://bugs.webkit.org/show_bug.cgi?id=225958
<rdar://78130303>

Reviewed by Eric Carlson.

* media/media-source/media-source-has-audio-video-expected.txt: Added.
* media/media-source/media-source-has-audio-video.html: Added.

2021-05-18 Cameron McCormack <heycam@apple.com>

Record gradient and pattern filled canvas text in the correct coordinate system.
@@ -0,0 +1,11 @@

RUN(video.src = URL.createObjectURL(mediaSource))
EVENT(sourceopen)
RUN(mediaSource.duration = loader.duration())
RUN(sourceBuffer = mediaSource.addSourceBuffer(loader.type()))
RUN(sourceBuffer.appendBuffer(loader.initSegment()))
EVENT(update)
EXPECTED (internals.mediaUsageState(video).hasVideo == 'true') OK
EXPECTED (internals.mediaUsageState(video).hasAudio == 'true') OK
END OF TEST

@@ -0,0 +1,43 @@
<!DOCTYPE html>
<html>
<head>
<title>media-source-has-audio-video</title>
<script src="media-source-loader.js"></script>
<script src="../video-test.js"></script>
<script>
window.addEventListener('load', event => {
findMediaElement();

window.sourceBuffer = new MediaSource();

loader = new MediaSourceLoader('content/test-fragmented-manifest.json');
loader.onload = mediaDataLoaded;
loader.onerror = mediaDataLoadingFailed;
});

function mediaDataLoadingFailed() {
failTest('Media data loading failed');
}

async function mediaDataLoaded() {
window.mediaSource = new MediaSource();
run('video.src = URL.createObjectURL(mediaSource)');
waitFor(video, 'error').then(failTest);
await waitFor(mediaSource, 'sourceopen');

run('mediaSource.duration = loader.duration()');
run('sourceBuffer = mediaSource.addSourceBuffer(loader.type())');
run('sourceBuffer.appendBuffer(loader.initSegment())');
await waitFor(sourceBuffer, 'update');

await testExpectedEventually('internals.mediaUsageState(video).hasVideo', true);
await testExpectedEventually('internals.mediaUsageState(video).hasAudio', true);
endTest();
}

</script>
</head>
<body>
<video></video>
</body>
</html>
@@ -1,3 +1,29 @@
2021-05-19 Jer Noble <jer.noble@apple.com>

[Mac] MSE-based video pauses when put in background; PiP context menu disabled
https://bugs.webkit.org/show_bug.cgi?id=225958
<rdar://78130303>

Reviewed by Eric Carlson.

Test: media/media-source/media-source-has-audio-video.html

In r277116, in order to reduce the CPU usage incurred when pushing state changes from
the GPU process to the WebContent process, state changes to `hasAudio` and `hasVideo`
were now only updated when the MediaPlayer indicated that its `characteristicsChanged()`.
However, in SourceBufferPrivateAVFObjC, this call was performed too early, before the
SourceBuffer in the WebContent process had informed SourceBufferPrivate of newly added
audio and video tracks. Thus the cached values of `hasAudio` and `hasVideo` inside the
WebContent process were never updated.

Call `characteristicsChanged()` from the callback handler of `didReceiveInitializationSegment()`,
which will be called only after the WebContent process completes its handling of the
new initialization segment, which includes telling the SourceBufferPrivate in the GPU
process about newly added audio and video tracks.

* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData):

2021-05-19 Antti Koivisto <antti@apple.com>

Factor pseudo class invalidation code in Document::updateHoverActiveState into a lambda
@@ -409,6 +409,9 @@ static void bufferWasConsumedCallback(CMNotificationCenterRef, const void* liste
if (!weakThis)
return;

if (auto player = this->player())
player->characteristicsChanged();

if (m_mediaSamples.isEmpty()) {
m_initializationSegmentIsHandled = true;
ALWAYS_LOG(LOGIDENTIFIER, "initialization segment is handled");

0 comments on commit 4c7f38a

Please sign in to comment.