Skip to content
Permalink
Browse files
[MSE][Mac] Support painting MSE video-element to canvas
https://bugs.webkit.org/show_bug.cgi?id=125157
<rdar://problem/23062016>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/media-source/media-source-paint-to-canvas.html

In order to have access to decoded video data for painting, decode the encoded samples manually
instead of adding them to the AVSampleBufferDisplayLayer. To facilitate doing so, add a new
utility class WebCoreDecompressionSession, which can decode samples and store them.

For the purposes of this patch, to avoid double-decoding of video data and to avoid severe complication
of our sample delivery pipeline, we will only support painting of decoded video samples when the video is
not displayed in the DOM.

* Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::seekToTime): Always send waitForSeekCompleted() to give private a chance to delay seek completion.
* Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::sourceBufferPrivateReenqueSamples): Added.
* Modules/mediasource/SourceBuffer.h:
* WebCore.xcodeproj/project.pbxproj:
* platform/cf/CoreMediaSoftLink.cpp: Added new soft link macros.
* platform/cf/CoreMediaSoftLink.h: Ditto.
* platform/cocoa/CoreVideoSoftLink.cpp: Ditto.
* platform/cocoa/CoreVideoSoftLink.h: Ditto.
* platform/graphics/SourceBufferPrivateClient.h:
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferDisplayLayer): Simple accessor.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession): Ditto.
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load): Update whether we should be displaying in a layer or decompression session..
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVisible): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): m_seeking is now an enum.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted): Ditto. If waiting for a video frame, delay completing seek.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime): Call updateLastImage() and return result.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage): Fetch the image for the current time.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paint): Pass to paintCurrentFrameInCanvas.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext): Get a native image, and render it.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): Create or destroy a layer or decompression session as appropriate.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): Creates a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer): Destroys a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession): Creates a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession): Destroys a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame): If seek completion delayed, complete now. Ditto for ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setReadyState): If waiting for a video frame, delay ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer): Deleted.
* platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
* platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::hasVideo): Promote to a class function.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideo): Return whether any of the active source buffers have video and are selected.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideoChanged): Call setSourceBufferWithSelectedVideo().
(WebCore::MediaSourcePrivateAVFObjC::setVideoLayer): Set (or clear) the layer on the selected buffer.
(WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Ditto for decompression session.
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo): Remove the layer and decompression session from the unselected

        buffer and add the decompression session or layer to the newly selected buffer.
(WebCore::MediaSourcePrivateAVFObjCHasVideo): Deleted.
* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::destroyRenderers): Clear the videoLayer and decompressionSession.
(WebCore::SourceBufferPrivateAVFObjC::hasSelectedVideo): Return whether the buffer has a selected video track.
(WebCore::SourceBufferPrivateAVFObjC::trackDidChangeEnabled): The media player now manages the video layer and decompression session lifetimes.
(WebCore::SourceBufferPrivateAVFObjC::flush): Flush the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample): Enqueue to the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples): As the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples): Tell the decompression session to stop requesting data, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples): Request media data from the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::setVideoLayer): Added.
(WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Added.
* platform/graphics/cocoa/WebCoreDecompressionSession.h: Added.
(WebCore::WebCoreDecompressionSession::create):
(WebCore::WebCoreDecompressionSession::isInvalidated):
(WebCore::WebCoreDecompressionSession::createWeakPtr):
* platform/graphics/cocoa/WebCoreDecompressionSession.mm: Added.
(WebCore::WebCoreDecompressionSession::WebCoreDecompressionSession): Register for media data requests.
(WebCore::WebCoreDecompressionSession::invalidate):  Unregister for same.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaDataCallback): Pass to maybeBecomeReadyForMoreMediaData.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData): Check in-flight decodes, and decoded frame counts.
(WebCore::WebCoreDecompressionSession::enqueueSample): Pass the sample to be decoded on a background queue.
(WebCore::WebCoreDecompressionSession::decodeSample): Decode the sample.
(WebCore::WebCoreDecompressionSession::decompressionOutputCallback): Call handleDecompressionOutput.
(WebCore::WebCoreDecompressionSession::handleDecompressionOutput): Pass decoded sample to be enqueued on the main thread.
(WebCore::WebCoreDecompressionSession::getFirstVideoFrame):
(WebCore::WebCoreDecompressionSession::enqueueDecodedSample): Enqueue the frame (if it's a displayed frame).
(WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData): Return whether we've hit our high water sample count.
(WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady):
(WebCore::WebCoreDecompressionSession::stopRequestingMediaData): Unset the same.
(WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame): Set a callback to notify when a decoded frame has been enqueued.
(WebCore::WebCoreDecompressionSession::imageForTime): Successively dequeue images until reaching one at or beyond the requested time.
(WebCore::WebCoreDecompressionSession::flush): Synchronously empty the producer and consumer queues.
(WebCore::WebCoreDecompressionSession::getDecodeTime): Utility method.
(WebCore::WebCoreDecompressionSession::getPresentationTime): Ditto.
(WebCore::WebCoreDecompressionSession::getDuration): Ditto.
(WebCore::WebCoreDecompressionSession::compareBuffers): Ditto.
* platform/cocoa/VideoToolboxSoftLink.cpp: Added.
* platform/cocoa/VideoToolboxSoftLink.h: Added.

LayoutTests:

* media/media-source/content/test-fragmented.mp4: Add a 'edts' atom to move the presentation time for the
    first sample to 0:00.
* media/media-source/content/test-fragmented-manifest.json:
* media/media-source/media-source-paint-to-canvas-expected.txt: Added.
* media/media-source/media-source-paint-to-canvas.html: Added.


Canonical link: https://commits.webkit.org/189312@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@217185 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
jernoble committed May 20, 2017
1 parent 49369eb commit b48e465b193cc203225ade596b49ae6553e22a53
Showing 25 changed files with 1,339 additions and 127 deletions.
@@ -1,3 +1,17 @@
2017-05-20 Jer Noble <jer.noble@apple.com>

[MSE][Mac] Support painting MSE video-element to canvas
https://bugs.webkit.org/show_bug.cgi?id=125157
<rdar://problem/23062016>

Reviewed by Eric Carlson.

* media/media-source/content/test-fragmented.mp4: Add a 'edts' atom to move the presentation time for the
first sample to 0:00.
* media/media-source/content/test-fragmented-manifest.json:
* media/media-source/media-source-paint-to-canvas-expected.txt: Added.
* media/media-source/media-source-paint-to-canvas.html: Added.

2017-05-19 Chris Dumez <cdumez@apple.com>

Consider not exposing webkitURL in workers
@@ -362,19 +376,6 @@

* media/media-source/content/test-fragmented.mp4:

2017-04-11 Jer Noble <jer.noble@apple.com>

[MSE][Mac] Support painting MSE video-element to canvas
https://bugs.webkit.org/show_bug.cgi?id=125157
<rdar://problem/23062016>

Reviewed by Eric Carlson.

* media/media-source/content/test-fragmented.mp4: Add a 'edts' atom to move the presentation time for the
first sample to 0:00.
* media/media-source/media-source-paint-to-canvas-expected.txt: Added.
* media/media-source/media-source-paint-to-canvas.html: Added.

2017-05-19 Zan Dobersek <zdobersek@igalia.com>

Unreviewed GTK+ gardening.
@@ -1,18 +1,18 @@
{
"url": "content/test-fragmented.mp4",
"type": "video/mp4; codecs=\"mp4a.40.2,avc1.4d281e\"",
"init": { "offset": 0, "size": 1231 },
"duration": 10.327753,
"init": { "offset": 0, "size": 1259 },
"duration": 10,
"media": [
{ "offset": 1231, "size": 67526, "timecode": 0.000000, "duration": 1.041668 },
{ "offset": 68757, "size": 72683, "timecode": 1.016916, "duration": 1.024752 },
{ "offset": 141440, "size": 78499, "timecode": 2.015374, "duration": 1.026294 },
{ "offset": 219939, "size": 77358, "timecode": 3.013832, "duration": 1.027835 },
{ "offset": 297297, "size": 80748, "timecode": 4.012290, "duration": 1.029377 },
{ "offset": 378045, "size": 78038, "timecode": 5.010748, "duration": 1.030919 },
{ "offset": 456083, "size": 82223, "timecode": 6.009206, "duration": 1.032461 },
{ "offset": 538306, "size": 78331, "timecode": 7.007664, "duration": 1.034003 },
{ "offset": 616637, "size": 80736, "timecode": 8.006122, "duration": 1.035545 },
{ "offset": 697373, "size": 77752, "timecode": 9.004580, "duration": 1.044899 }
{ "offset": 1259, "size": 67526, "timestamp": 0, "duration": 1 },
{ "offset": 68785, "size": 72683, "timestamp": 1, "duration": 1 },
{ "offset": 141468, "size": 78499, "timestamp": 2, "duration": 1 },
{ "offset": 219967, "size": 77358, "timestamp": 3, "duration": 1 },
{ "offset": 297325, "size": 80748, "timestamp": 4, "duration": 1 },
{ "offset": 378073, "size": 78038, "timestamp": 5, "duration": 1 },
{ "offset": 456111, "size": 82223, "timestamp": 6, "duration": 1 },
{ "offset": 538334, "size": 78331, "timestamp": 7, "duration": 1 },
{ "offset": 616665, "size": 80736, "timestamp": 8, "duration": 1 },
{ "offset": 697401, "size": 77752, "timestamp": 9, "duration": 1 }
]
}
Binary file not shown.
@@ -0,0 +1,14 @@
EVENT(sourceopen)
EVENT(update)
EVENT(canplay)
RUN(video.currentTime += 1.001 / 24)
EVENT(seeked)
EXPECTED (canvas.getContext("2d").getImageData(250, 130, 1, 1).data[0] != '0') OK
RUN(video.currentTime += 1.001 / 24)
EVENT(seeked)
EXPECTED (canvas.getContext("2d").getImageData(250, 130, 1, 1).data[0] != '0') OK
RUN(video.currentTime += 1.001 / 24)
EVENT(seeked)
EXPECTED (canvas.getContext("2d").getImageData(250, 130, 1, 1).data[0] != '0') OK
END OF TEST

@@ -0,0 +1,79 @@
<!DOCTYPE html>
<html>
<head>
<title>media-source-stalled-holds-sleep-assertion</title>
<script src="media-source-loader.js"></script>
<script src="../video-test.js"></script>
<script>

var canvas;
var loader;
var source;
var sourceBuffer;

function runTest() {
mediaElement = video = document.createElement('video');

loader = new MediaSourceLoader('content/test-fragmented-manifest.json');
loader.onload = mediaDataLoaded;
loader.onerror = mediaDataLoadingFailed;
}

function mediaDataLoadingFailed() {
failTest('Media data loading failed');
}

function mediaDataLoaded() {
source = new MediaSource();
waitForEvent('sourceopen', sourceOpen, false, false, source);
video.src = URL.createObjectURL(source);
}

function sourceOpen() {
source.duration = loader.duration();
sourceBuffer = source.addSourceBuffer(loader.type());
waitForEventOn(sourceBuffer, 'update', sourceInitialized, false, true);
sourceBuffer.appendBuffer(loader.initSegment());
}

function sourceInitialized() {
waitForEvent('canplay', canPlay, false, true);
sourceBuffer.appendBuffer(loader.mediaSegment(0));
}

function paint() {
canvas = document.createElement('canvas');
canvas.width = video.videoWidth / 2;
canvas.height = video.videoHeight / 2;
canvas.getContext('2d').drawImage(video, 0, 0, canvas.width, canvas.height);
document.getElementById('canvases').appendChild(canvas);
testExpected('canvas.getContext("2d").getImageData(250, 130, 1, 1).data[0]', '0', '!=');
}

function canPlay() {
waitForEvent('seeked', seeked1, false, true);
run('video.currentTime += 1.001 / 24');
}

function seeked1() {
paint();
waitForEvent('seeked', seeked2, false, true);
run('video.currentTime += 1.001 / 24');
}

function seeked2() {
paint();
waitForEvent('seeked', seeked3, false, true);
run('video.currentTime += 1.001 / 24');
}

function seeked3() {
paint();
endTest();
}
</script>
</head>
<body onload="runTest()">
<div id="canvases"></canvas>
</body>
</html>
@@ -1,3 +1,106 @@
2017-05-20 Jer Noble <jer.noble@apple.com>

[MSE][Mac] Support painting MSE video-element to canvas
https://bugs.webkit.org/show_bug.cgi?id=125157
<rdar://problem/23062016>

Reviewed by Eric Carlson.

Test: media/media-source/media-source-paint-to-canvas.html

In order to have access to decoded video data for painting, decode the encoded samples manually
instead of adding them to the AVSampleBufferDisplayLayer. To facilitate doing so, add a new
utility class WebCoreDecompressionSession, which can decode samples and store them.

For the purposes of this patch, to avoid double-decoding of video data and to avoid severe complication
of our sample delivery pipeline, we will only support painting of decoded video samples when the video is
not displayed in the DOM.

* Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::seekToTime): Always send waitForSeekCompleted() to give private a chance to delay seek completion.
* Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::sourceBufferPrivateReenqueSamples): Added.
* Modules/mediasource/SourceBuffer.h:
* WebCore.xcodeproj/project.pbxproj:
* platform/cf/CoreMediaSoftLink.cpp: Added new soft link macros.
* platform/cf/CoreMediaSoftLink.h: Ditto.
* platform/cocoa/CoreVideoSoftLink.cpp: Ditto.
* platform/cocoa/CoreVideoSoftLink.h: Ditto.
* platform/graphics/SourceBufferPrivateClient.h:
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::sampleBufferDisplayLayer): Simple accessor.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession): Ditto.
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load): Update whether we should be displaying in a layer or decompression session..
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVisible): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): m_seeking is now an enum.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking): Ditto.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted): Ditto. If waiting for a video frame, delay completing seek.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::nativeImageForCurrentTime): Call updateLastImage() and return result.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastImage): Fetch the image for the current time.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paint): Pass to paintCurrentFrameInCanvas.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::paintCurrentFrameInContext): Get a native image, and render it.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): Create or destroy a layer or decompression session as appropriate.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): Creates a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer): Destroys a layer.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession): Creates a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession): Destroys a decompression session.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame): If seek completion delayed, complete now. Ditto for ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setReadyState): If waiting for a video frame, delay ready state change.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::addDisplayLayer): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::removeDisplayLayer): Deleted.
* platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
* platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::hasVideo): Promote to a class function.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideo): Return whether any of the active source buffers have video and are selected.
(WebCore::MediaSourcePrivateAVFObjC::hasSelectedVideoChanged): Call setSourceBufferWithSelectedVideo().
(WebCore::MediaSourcePrivateAVFObjC::setVideoLayer): Set (or clear) the layer on the selected buffer.
(WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Ditto for decompression session.
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo): Remove the layer and decompression session from the unselected

buffer and add the decompression session or layer to the newly selected buffer.
(WebCore::MediaSourcePrivateAVFObjCHasVideo): Deleted.
* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::destroyRenderers): Clear the videoLayer and decompressionSession.
(WebCore::SourceBufferPrivateAVFObjC::hasSelectedVideo): Return whether the buffer has a selected video track.
(WebCore::SourceBufferPrivateAVFObjC::trackDidChangeEnabled): The media player now manages the video layer and decompression session lifetimes.
(WebCore::SourceBufferPrivateAVFObjC::flush): Flush the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample): Enqueue to the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples): As the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples): Tell the decompression session to stop requesting data, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples): Request media data from the decompression session, if it exists.
(WebCore::SourceBufferPrivateAVFObjC::setVideoLayer): Added.
(WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Added.
* platform/graphics/cocoa/WebCoreDecompressionSession.h: Added.
(WebCore::WebCoreDecompressionSession::create):
(WebCore::WebCoreDecompressionSession::isInvalidated):
(WebCore::WebCoreDecompressionSession::createWeakPtr):
* platform/graphics/cocoa/WebCoreDecompressionSession.mm: Added.
(WebCore::WebCoreDecompressionSession::WebCoreDecompressionSession): Register for media data requests.
(WebCore::WebCoreDecompressionSession::invalidate): Unregister for same.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaDataCallback): Pass to maybeBecomeReadyForMoreMediaData.
(WebCore::WebCoreDecompressionSession::maybeBecomeReadyForMoreMediaData): Check in-flight decodes, and decoded frame counts.
(WebCore::WebCoreDecompressionSession::enqueueSample): Pass the sample to be decoded on a background queue.
(WebCore::WebCoreDecompressionSession::decodeSample): Decode the sample.
(WebCore::WebCoreDecompressionSession::decompressionOutputCallback): Call handleDecompressionOutput.
(WebCore::WebCoreDecompressionSession::handleDecompressionOutput): Pass decoded sample to be enqueued on the main thread.
(WebCore::WebCoreDecompressionSession::getFirstVideoFrame):
(WebCore::WebCoreDecompressionSession::enqueueDecodedSample): Enqueue the frame (if it's a displayed frame).
(WebCore::WebCoreDecompressionSession::isReadyForMoreMediaData): Return whether we've hit our high water sample count.
(WebCore::WebCoreDecompressionSession::requestMediaDataWhenReady):
(WebCore::WebCoreDecompressionSession::stopRequestingMediaData): Unset the same.
(WebCore::WebCoreDecompressionSession::notifyWhenHasAvailableVideoFrame): Set a callback to notify when a decoded frame has been enqueued.
(WebCore::WebCoreDecompressionSession::imageForTime): Successively dequeue images until reaching one at or beyond the requested time.
(WebCore::WebCoreDecompressionSession::flush): Synchronously empty the producer and consumer queues.
(WebCore::WebCoreDecompressionSession::getDecodeTime): Utility method.
(WebCore::WebCoreDecompressionSession::getPresentationTime): Ditto.
(WebCore::WebCoreDecompressionSession::getDuration): Ditto.
(WebCore::WebCoreDecompressionSession::compareBuffers): Ditto.
* platform/cocoa/VideoToolboxSoftLink.cpp: Added.
* platform/cocoa/VideoToolboxSoftLink.h: Added.

2017-05-19 Joseph Pecoraro <pecoraro@apple.com>

WebAVStreamDataParserListener String leak
@@ -233,6 +233,7 @@ void MediaSource::seekToTime(const MediaTime& time)
// ↳ Otherwise
// Continue

m_private->waitForSeekCompleted();
completeSeek();
}

@@ -1778,8 +1778,26 @@ void SourceBuffer::textTrackKindChanged(TextTrack& track)
m_source->mediaElement()->textTrackKindChanged(track);
}

void SourceBuffer::sourceBufferPrivateReenqueSamples(const AtomicString& trackID)
{
if (isRemoved())
return;

LOG(MediaSource, "SourceBuffer::sourceBufferPrivateReenqueSamples(%p)", this);
auto it = m_trackBufferMap.find(trackID);
if (it == m_trackBufferMap.end())
return;

auto& trackBuffer = it->value;
trackBuffer.needsReenqueueing = true;
reenqueueMediaForTime(trackBuffer, trackID, m_source->currentTime());
}

void SourceBuffer::sourceBufferPrivateDidBecomeReadyForMoreSamples(const AtomicString& trackID)
{
if (isRemoved())
return;

LOG(MediaSource, "SourceBuffer::sourceBufferPrivateDidBecomeReadyForMoreSamples(%p)", this);
auto it = m_trackBufferMap.find(trackID);
if (it == m_trackBufferMap.end())
@@ -1930,6 +1948,9 @@ void SourceBuffer::updateBufferedFromTrackBuffers()
for (auto& trackBuffer : m_trackBufferMap.values()) {
// 4.1 Let track ranges equal the track buffer ranges for the current track buffer.
PlatformTimeRanges trackRanges = trackBuffer.buffered;
if (!trackRanges.length())
continue;

// 4.2 If readyState is "ended", then set the end time on the last range in track ranges to highest end time.
if (m_source->isEnded())
trackRanges.add(trackRanges.maximumBufferedTime(), highestEndTime);
@@ -129,6 +129,7 @@ class SourceBuffer final : public RefCounted<SourceBuffer>, public ActiveDOMObje
void sourceBufferPrivateDidReceiveSample(MediaSample&) final;
bool sourceBufferPrivateHasAudio() const final;
bool sourceBufferPrivateHasVideo() const final;
void sourceBufferPrivateReenqueSamples(const AtomicString& trackID) final;
void sourceBufferPrivateDidBecomeReadyForMoreSamples(const AtomicString& trackID) final;
MediaTime sourceBufferPrivateFastSeekTimeForMediaTime(const MediaTime&, const MediaTime& negativeThreshold, const MediaTime& positiveThreshold) final;
void sourceBufferPrivateAppendComplete(AppendResult) final;

0 comments on commit b48e465

Please sign in to comment.