Skip to content
Permalink
Browse files
[WPE] Add support for holepunch using an external video player
https://bugs.webkit.org/show_bug.cgi?id=194899

Reviewed by Xabier Rodriguez-Calvar.

.:

Add EXTERNAL_HOLEPUNCH option to the WPE port. Add a manual test to check the
feature. Rename and update the test for GStreamer holepunch to avoid confusion.

* ManualTests/wpe/video-player-holepunch-external.html: Added.
* ManualTests/wpe/video-player-holepunch-gstreamer.html: Renamed from ManualTests/wpe/video-player-holepunch.html.
* Source/cmake/OptionsWPE.cmake:

Source/WebCore:

Implement the holepunch feature to allow playback using an external player. This creates
a new dummy MediaPlayerPrivateHolePunch that accepts only the type "video/holepunch", and
whose goal is to just draw a transparent rectangle in the position where the video should be.
This can be used to allow a player placed on a lower plane than the browser to become visible.

Added ManualTest wpe/video-player-holepunch-external.html to test the feature.

* PlatformWPE.cmake:
* platform/HolePunch.cmake: Added.
* platform/graphics/MediaPlayer.cpp:
(WebCore::buildMediaEnginesVector):
* platform/graphics/holepunch/MediaPlayerPrivateHolePunch.cpp: Added.
(WebCore::MediaPlayerPrivateHolePunch::MediaPlayerPrivateHolePunch):
(WebCore::MediaPlayerPrivateHolePunch::~MediaPlayerPrivateHolePunch):
(WebCore::MediaPlayerPrivateHolePunch::platformLayer const):
(WebCore::MediaPlayerPrivateHolePunch::naturalSize const):
(WebCore::MediaPlayerPrivateHolePunch::pushNextHolePunchBuffer):
(WebCore::MediaPlayerPrivateHolePunch::swapBuffersIfNeeded):
(WebCore::MediaPlayerPrivateHolePunch::proxy const):
(WebCore::mimeTypeCache):
(WebCore::MediaPlayerPrivateHolePunch::getSupportedTypes):
(WebCore::MediaPlayerPrivateHolePunch::supportsType):
(WebCore::MediaPlayerPrivateHolePunch::registerMediaEngine):
(WebCore::MediaPlayerPrivateHolePunch::notifyReadyState):
* platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h: Added.
* platform/graphics/texmap/TextureMapperPlatformLayerBuffer.cpp:
(WebCore::TextureMapperPlatformLayerBuffer::paintToTextureMapper):


Canonical link: https://commits.webkit.org/209357@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@242033 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
magomez committed Feb 25, 2019
1 parent 8cc3390 commit 4df2ab73eb8d2f6585ae74e7c4b4ba2fd19096af
Showing 11 changed files with 404 additions and 5 deletions.
@@ -1,3 +1,17 @@
2019-02-25 Miguel Gomez <magomez@igalia.com>

[WPE] Add support for holepunch using an external video player
https://bugs.webkit.org/show_bug.cgi?id=194899

Reviewed by Xabier Rodriguez-Calvar.

Add EXTERNAL_HOLEPUNCH option to the WPE port. Add a manual test to check the
feature. Rename and update the test for GStreamer holepunch to avoid confusion.

* ManualTests/wpe/video-player-holepunch-external.html: Added.
* ManualTests/wpe/video-player-holepunch-gstreamer.html: Renamed from ManualTests/wpe/video-player-holepunch.html.
* Source/cmake/OptionsWPE.cmake:

2019-02-24 Devin Rousso <drousso@apple.com>

Web Inspector: Change the InspectorOverlay to use native rather than canvas
@@ -0,0 +1,56 @@
<html>
<head>
<title>WPE external holepunch test</title>
<style>
video {
width: 400px;
height: 400px;
}
</style>
</head>
<body>
<p>
<strong>WPE external holepunch test</strong>
</p>
<p>
This test checks whether the external holepunch feature is working on WPE.
</p>
<p>
There is a video player of 400x400 below this text. It will play a valid video for 5 seconds, then a holepunch one
for another 5 seconds and then the first one again.
</p>
<p>
If everything is working fine, there should be the valid video for 5 seconds, then a transparent rectangle of 400x400
for 5 seconds, and then again the valid video.
</p>
<p>
If GStreamer holepunch is enabled when running this test, the first and third video will show a transparent
rectangle instead of the video content.
</p>
<video id="video">
<source id="source" src="../../LayoutTests/media/content/long-test.mp4" type="video/mp4">
</video>
<div id="log"></div>
<script type="text/javascript">
var v = document.getElementById("video");
var s = document.getElementById("source");
var l = document.getElementById("log");
log.innerHTML = "Playing first video. The video (or a transparent rectangle if GStreamer holepunch is enabled) should be visible";
v.play();

setTimeout(function() {
s.type = "video/holepunch";
v.load();
v.play();
log.innerHTML = "Playing second video. A transparent rectangle should be visible";
}, 5000);

setTimeout(function() {
s.type = "video/mp4";
v.load();
v.play();
log.innerHTML = "Playing third video. The video (or a transparent rectangle if GStreamer holepunch is enabled) should be visible";
}, 10000);
</script>
</body>
</html>
@@ -1,6 +1,6 @@
<html>
<head>
<title>WPE holepunch test</title>
<title>WPE GStreamer holepunch test</title>
<style>
video {
width: 400px;
@@ -10,10 +10,10 @@
</head>
<body>
<p>
<strong>WPE holepunch test</strong>
<strong>WPE GStreamer holepunch test</strong>
</p>
<p>
This test checks whether the holepunch feature is working on WPE.
This test checks whether the GStreamer holepunch feature is working on WPE.
</p>
<p>
There is a video player of 400x400 placed below this text. If the video playback is visible,
@@ -1,3 +1,38 @@
2019-02-25 Miguel Gomez <magomez@igalia.com>

[WPE] Add support for holepunch using an external video player
https://bugs.webkit.org/show_bug.cgi?id=194899

Reviewed by Xabier Rodriguez-Calvar.

Implement the holepunch feature to allow playback using an external player. This creates
a new dummy MediaPlayerPrivateHolePunch that accepts only the type "video/holepunch", and
whose goal is to just draw a transparent rectangle in the position where the video should be.
This can be used to allow a player placed on a lower plane than the browser to become visible.

Added ManualTest wpe/video-player-holepunch-external.html to test the feature.

* PlatformWPE.cmake:
* platform/HolePunch.cmake: Added.
* platform/graphics/MediaPlayer.cpp:
(WebCore::buildMediaEnginesVector):
* platform/graphics/holepunch/MediaPlayerPrivateHolePunch.cpp: Added.
(WebCore::MediaPlayerPrivateHolePunch::MediaPlayerPrivateHolePunch):
(WebCore::MediaPlayerPrivateHolePunch::~MediaPlayerPrivateHolePunch):
(WebCore::MediaPlayerPrivateHolePunch::platformLayer const):
(WebCore::MediaPlayerPrivateHolePunch::naturalSize const):
(WebCore::MediaPlayerPrivateHolePunch::pushNextHolePunchBuffer):
(WebCore::MediaPlayerPrivateHolePunch::swapBuffersIfNeeded):
(WebCore::MediaPlayerPrivateHolePunch::proxy const):
(WebCore::mimeTypeCache):
(WebCore::MediaPlayerPrivateHolePunch::getSupportedTypes):
(WebCore::MediaPlayerPrivateHolePunch::supportsType):
(WebCore::MediaPlayerPrivateHolePunch::registerMediaEngine):
(WebCore::MediaPlayerPrivateHolePunch::notifyReadyState):
* platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h: Added.
* platform/graphics/texmap/TextureMapperPlatformLayerBuffer.cpp:
(WebCore::TextureMapperPlatformLayerBuffer::paintToTextureMapper):

2019-02-24 Zalan Bujtas <zalan@apple.com>

Introduce ContentChangeObserver class
@@ -5,6 +5,10 @@ include(platform/GStreamer.cmake)
include(platform/ImageDecoders.cmake)
include(platform/TextureMapper.cmake)

if (USE_EXTERNAL_HOLEPUNCH)
include(platform/HolePunch.cmake)
endif ()

list(APPEND WebCore_UNIFIED_SOURCE_LIST_FILES
"SourcesWPE.txt"

@@ -0,0 +1,6 @@
list(APPEND WebCore_INCLUDE_DIRECTORIES
"${WEBCORE_DIR}/platform/graphics/holepunch"
)
list(APPEND WebCore_SOURCES
platform/graphics/holepunch/MediaPlayerPrivateHolePunch.cpp
)
@@ -84,6 +84,10 @@
#include "MediaPlayerPrivateAVFoundationCF.h"
#endif

#if USE(EXTERNAL_HOLEPUNCH)
#include "MediaPlayerPrivateHolePunch.h"
#endif

namespace WebCore {

#if !RELEASE_LOG_DISABLED
@@ -255,6 +259,10 @@ static void buildMediaEnginesVector()
MediaPlayerPrivateGStreamerMSE::registerMediaEngine(addMediaEngine);
#endif

#if USE(EXTERNAL_HOLEPUNCH)
MediaPlayerPrivateHolePunch::registerMediaEngine(addMediaEngine);
#endif

haveMediaEnginesVector() = true;
}

@@ -0,0 +1,154 @@
/*
* Copyright (C) 2019 Igalia S.L
* Copyright (C) 2019 Metrological Group B.V.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public License
* aint with this library; see the file COPYING.LIB. If not, write to
* the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/

#include "config.h"
#include "MediaPlayerPrivateHolePunch.h"

#if USE(EXTERNAL_HOLEPUNCH)
#include "MediaPlayer.h"
#include "TextureMapperPlatformLayerBuffer.h"
#include "TextureMapperPlatformLayerProxy.h"

namespace WebCore {

static const FloatSize s_holePunchDefaultFrameSize(1280, 720);

MediaPlayerPrivateHolePunch::MediaPlayerPrivateHolePunch(MediaPlayer* player)
: m_player(player)
, m_readyTimer(RunLoop::main(), this, &MediaPlayerPrivateHolePunch::notifyReadyState)
#if USE(NICOSIA)
, m_nicosiaLayer(Nicosia::ContentLayer::create(Nicosia::ContentLayerTextureMapperImpl::createFactory(*this)))
#else
, m_platformLayerProxy(adoptRef(new TextureMapperPlatformLayerProxy()))
#endif
{
pushNextHolePunchBuffer();

// Delay the configuration of the HTMLMediaElement, as during this stage this is not
// the MediaPlayer private yet and calls from HTMLMediaElement won't reach this.
m_readyTimer.startOneShot(0_s);
}

MediaPlayerPrivateHolePunch::~MediaPlayerPrivateHolePunch()
{
#if USE(NICOSIA)
downcast<Nicosia::ContentLayerTextureMapperImpl>(m_nicosiaLayer->impl()).invalidateClient();
#endif
}

PlatformLayer* MediaPlayerPrivateHolePunch::platformLayer() const
{
#if USE(NICOSIA)
return m_nicosiaLayer.ptr();
#else
return const_cast<MediaPlayerPrivateHolePunch*>(this);
#endif
}

FloatSize MediaPlayerPrivateHolePunch::naturalSize() const
{
// When using the holepuch we may not be able to get the video frames size, so we can't use
// it. But we need to report some non empty naturalSize for the player's GraphicsLayer
// to be properly created.
return s_holePunchDefaultFrameSize;
}

void MediaPlayerPrivateHolePunch::pushNextHolePunchBuffer()
{
auto proxyOperation =
[this](TextureMapperPlatformLayerProxy& proxy)
{
LockHolder holder(proxy.lock());
std::unique_ptr<TextureMapperPlatformLayerBuffer> layerBuffer = std::make_unique<TextureMapperPlatformLayerBuffer>(0, m_size, TextureMapperGL::ShouldNotBlend, GL_DONT_CARE);
proxy.pushNextBuffer(WTFMove(layerBuffer));
};

#if USE(NICOSIA)
proxyOperation(downcast<Nicosia::ContentLayerTextureMapperImpl>(m_nicosiaLayer->impl()).proxy());
#else
proxyOperation(*m_platformLayerProxy);
#endif
}

void MediaPlayerPrivateHolePunch::swapBuffersIfNeeded()
{
pushNextHolePunchBuffer();
}

#if !USE(NICOSIA)
RefPtr<TextureMapperPlatformLayerProxy> MediaPlayerPrivateHolePunch::proxy() const
{
return m_platformLayerProxy.copyRef();
}
#endif

static HashSet<String, ASCIICaseInsensitiveHash>& mimeTypeCache()
{
static NeverDestroyed<HashSet<String, ASCIICaseInsensitiveHash>> cache;
static bool typeListInitialized = false;

if (typeListInitialized)
return cache;

const char* mimeTypes[] = {
"video/holepunch"
};

for (unsigned i = 0; i < (sizeof(mimeTypes) / sizeof(*mimeTypes)); ++i)
cache.get().add(String(mimeTypes[i]));

typeListInitialized = true;

return cache;
}

void MediaPlayerPrivateHolePunch::getSupportedTypes(HashSet<String, ASCIICaseInsensitiveHash>& types)
{
types = mimeTypeCache();
}

MediaPlayer::SupportsType MediaPlayerPrivateHolePunch::supportsType(const MediaEngineSupportParameters& parameters)
{
auto containerType = parameters.type.containerType();

// Spec says we should not return "probably" if the codecs string is empty.
if (!containerType.isEmpty() && mimeTypeCache().contains(containerType)) {
if (parameters.type.codecs().isEmpty())
return MediaPlayer::MayBeSupported;

return MediaPlayer::IsSupported;
}

return MediaPlayer::IsNotSupported;
}

void MediaPlayerPrivateHolePunch::registerMediaEngine(MediaEngineRegistrar registrar)
{
registrar([](MediaPlayer* player) { return std::make_unique<MediaPlayerPrivateHolePunch>(player); },
getSupportedTypes, supportsType, nullptr, nullptr, nullptr, nullptr);
}

void MediaPlayerPrivateHolePunch::notifyReadyState()
{
// Notify the ready state so the GraphicsLayer gets created.
m_player->readyStateChanged();
}
}
#endif // USE(EXTERNAL_HOLEPUNCH)

0 comments on commit 4df2ab7

Please sign in to comment.