Eliminate double layer hosting of video content when UI-Side compositing is enabled #9910

jernoble · 2023-02-10T08:25:38Z

`897c865`

Eliminate double layer hosting of video content when UI-Side compositing is enabled
https://bugs.webkit.org/show_bug.cgi?id=252021
<rdar://105246033>

Reviewed by Eric Carlson and Per Arne Vollan.

Currently, video content from the GPU process is remotely-hosted into the WebContent
process, and is subsequently remotely hosted from the WebContent process to the UI
process when UI-side compositing is enabled. This is unnecessary, and necessitates
having an active CARenderer in the Web Content process. Rather than double hosting
video content from GPU -> WebContent -> UI process, video content should be hosted
directly from the GPU process into the UI process.

To facilitate this, re-use the infrastructure built into VideoFullscreenManager to
handle the required XPC communication to send resize information back down from the
UI process to the GPU process.

HTMLMediaElement will need to expose API to resize its video layer, and to provide
the remote hosting context ID from the GPU process.

Rather than have a separate identifier of its own, VideoFullscreenManager and
PlaybackSessionManager can just use a HTMLMediaElement identifier for the purposes
of passing messages across the XPC boundary.

WebAVPlayerLayer must support use cases more like VideoLayerRemoteCocoa, including
handling multiple VideoGravity settings, so it is improved to take these settings
into account.

WebAVPlayerLayerView has been pulled out of VideoFullScreenManagerAVKit into its
own source and header files.

GraphicsLayerCA, RemoteLayerTreeHost, and RemoteLayerTree all must be modified to
accept a HTMLVideoElement (or its identifier) as a parameter.

* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/dom/ImageOverlay.cpp:
(WebCore::ImageOverlay::updateSubtree):
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::setVideoInlineSizeFenced):
(WebCore::HTMLMediaElement::layerHostingContextID):
(WebCore::HTMLMediaElement::naturalSize):
* Source/WebCore/html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::identifier const):
* Source/WebCore/platform/cocoa/VideoFullscreenModel.h:
* Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.h:
* Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.mm:
(WebCore::VideoFullscreenModelVideoElement::setVideoInlineSizeFenced):
* Source/WebCore/platform/cocoa/WebAVPlayerLayer.h:
* Source/WebCore/platform/cocoa/WebAVPlayerLayer.mm:
(-[WebAVPlayerLayer layoutSublayers]):
(-[WebAVPlayerLayer resolveBounds]):
* Source/WebCore/platform/graphics/GraphicsLayer.h:
(WebCore::GraphicsLayer::layerMode const):
(WebCore::GraphicsLayer::setContentsToVideoElement):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::setVideoInlineSizeFenced):
(WebCore::MediaPlayer::hostingContextID const):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::setVideoInlineSizeFenced):
(WebCore::MediaPlayerPrivateInterface::hostingContextID const):
* Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:
(WebCore::GraphicsLayerCA::createPlatformCALayer):
(WebCore::GraphicsLayerCA::setContentsToPlatformLayer):
(WebCore::GraphicsLayerCA::setContentsToVideoElement):
* Source/WebCore/platform/graphics/ca/GraphicsLayerCA.h:
* Source/WebCore/platform/graphics/ca/PlatformCALayer.cpp:
(WebCore::operator<<):
* Source/WebCore/platform/graphics/ca/PlatformCALayer.h:
* Source/WebCore/platform/graphics/ca/cocoa/PlatformCALayerCocoa.mm:
(WebCore::PlatformCALayerCocoa::PlatformCALayerCocoa):
* Source/WebCore/platform/graphics/ca/cocoa/WebVideoContainerLayer.mm:
(-[WebVideoContainerLayer setPosition:]): Deleted.
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/ios/VideoFullscreenInterfaceAVKit.h:
* Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.h:
* Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.mm:
(-[WebVideoFullscreenInterfaceMacObjC setUpPIPForVideoView:withFrame:inWindow:]):
(WebCore::VideoFullscreenInterfaceMac::createLayer):
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateConfiguration):
* Source/WebKit/Platform/cocoa/LayerHostingContext.h:
* Source/WebKit/Platform/cocoa/LayerHostingContext.mm:
(WebKit::LayerHostingContext::createForPort):
(WebKit::LayerHostingContext::createForExternalHostingProcess):
(WebKit::LayerHostingContext::createTransportLayerForRemoteHosting):
(WebKit::LayerHostingContext::updateCachedContextID):
(WebKit::LayerHostingContext::cachedContextID):
* Source/WebKit/Shared/PlaybackSessionContextIdentifier.h:
(): Deleted.
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm:
(WebKit::RemoteLayerBackingStore::drawInContext):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h:
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm:
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode const):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode):
(WebKit::RemoteLayerTreeTransaction::description const):
* Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.h:
* Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.mm:
(WebKit::VideoFullscreenManagerProxy::createLayerWithID):
(WebKit::VideoFullscreenManagerProxy::createLayerHostViewWithID):
(WebKit::VideoFullscreenManagerProxy::setupFullscreenWithID):
(WebKit::VideoFullscreenManagerProxy::didCleanupFullscreen):
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm:
(WebKit::RemoteLayerTreeHost::makeNode):
* Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm:
(WebKit::RemoteLayerTreeHost::makeNode):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::hostingContextID const):
(WebKit::MediaPlayerPrivateRemote::setLayerHostingContextID):
(WebKit::MediaPlayerPrivateRemote::setCachedPresentationSize):
(WebKit::MediaPlayerPrivateRemote::cachedPresentationSize):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
(WebKit::MediaPlayerPrivateRemote::layerHostingContextIdChanged):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp:
(WebKit::GraphicsLayerCARemote::createPlatformCALayer):
(WebKit::GraphicsLayerCARemote::layerMode const):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp:
(WebKit::PlatformCALayerRemote::create):
(WebKit::PlatformCALayerRemote::recursiveBuildTransaction):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.mm:
(WebKit::PlatformCALayerRemoteCustom::create):
(WebKit::PlatformCALayerRemoteCustom::PlatformCALayerRemoteCustom):
(WebKit::PlatformCALayerRemoteCustom::hostingContextID):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.h:
(WebKit::RemoteLayerTreeContext::webPage):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm:
(WebKit::RemoteLayerTreeContext::layerDidEnterContext):
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::m_appHighlightsVisible):
* Source/WebKit/WebProcess/WebPage/mac/TiledCoreAnimationDrawingArea.mm:
(WebKit::TiledCoreAnimationDrawingArea::sendDidFirstLayerFlushIfNeeded):
(WebKit::TiledCoreAnimationDrawingArea::sendEnterAcceleratedCompositingModeIfNeeded):
(WebKit::TiledCoreAnimationDrawingArea::setLayerHostingMode):
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h:
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm:
(WebKit::PlaybackSessionManager::removeContext):
(WebKit::PlaybackSessionManager::setUpPlaybackControlsManager):
(WebKit::PlaybackSessionManager::contextIdForMediaElement):
* Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.h:
* Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.mm:
(WebKit::VideoFullscreenManager::setupRemoteLayerHosting):
(WebKit::VideoFullscreenManager::enterVideoFullscreenForVideoElement):
(WebKit::VideoFullscreenManager::setVideoLayerFrameFenced):

Co-authored-by: Arunsundar Kannan <arunsundar_kannan@apple.com>

Canonical link: https://commits.webkit.org/260575@main

89dd664

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	⏳ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	⏳ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 gtk-wk2
	⏳ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch	✅ 🧪 mac-wk2-stress
✅ 🛠 🧪 merge	✅ 🛠 watch-sim

webkit-early-warning-system · 2023-02-10T08:27:20Z

EWS run on previous version of this PR (hash 554a0ba)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	❌ 🛠 wpe	❌ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	❌ 🛠 gtk
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	✅ 🧪 api-mac	❌ 🧪 gtk-wk2
	❌ 🧪 api-ios	✅ 🧪 mac-wk1	❌ 🧪 api-gtk
	❌ 🛠 tv	✅ 🧪 mac-wk2	⏳ 🛠 jsc-armv7
	❌ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2	⏳ 🧪 jsc-armv7-tests
	❌ 🛠 watch	✅ 🧪 mac-wk2-stress
	❌ 🛠 watch-sim

webkit-early-warning-system · 2023-02-10T17:08:55Z

EWS run on previous version of this PR (hash 3416692)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	✅ 🧪 api-mac	✅ 🧪 gtk-wk2
	❌ 🧪 api-ios	✅ 🧪 mac-wk1	~~🧪 api-gtk~~
	❌ 🛠 tv	✅ 🧪 mac-wk2	⏳ 🛠 jsc-armv7
	❌ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2	⏳ 🧪 jsc-armv7-tests
	❌ 🛠 watch	✅ 🧪 mac-wk2-stress
	❌ 🛠 watch-sim

eric-carlson

r=me

eric-carlson · 2023-02-10T17:43:59Z

Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp

if (auto hostingContextID = videoElement.layerHostingContextID != m_layerHostingContextID) ?

I don't think you can initialize and check in the same if statement.

It is done a lot in WebKit

Actually you did it yourself in https://bugs.webkit.org/show_bug.cgi?id=162971

eric-carlson · 2023-02-10T17:45:21Z

Source/WebKit/Platform/cocoa/LayerHostingContext.h

Nit: Space between // and Denotes. This is really long and should be wrapped.

eric-carlson · 2023-02-10T17:51:44Z

Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.mm

Can this be moved inside of the if (auto playerLayer = model->playerLayer()) block?

eric-carlson · 2023-02-10T17:52:43Z

Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm

if (auto videoLayerIter = m_videoLayers.find(layerID) != m_videoLayers())

eric-carlson · 2023-02-10T18:00:36Z

Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm

if (auto videoLayerIter = m_videoLayers.find(layerID) != m_videoLayers())

webkit-early-warning-system · 2023-02-10T22:44:41Z

EWS run on previous version of this PR (hash f30834a)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	❌ 🛠 ios	❌ 🛠 mac	❌ 🛠 wpe	❌ 🛠 wincairo
❌ 🧪 bindings	❌ 🛠 ios-sim	❌ 🛠 mac-AS-debug	❌ 🛠 gtk
❌ 🧪 webkitperl	❌ 🧪 ios-wk2	❌ 🧪 api-mac	❌ 🧪 gtk-wk2
	❌ 🧪 api-ios	❌ 🧪 mac-wk1	❌ 🧪 api-gtk
	❌ 🛠 tv	❌ 🧪 mac-wk2
	❌ 🛠 tv-sim	❌ 🧪 mac-AS-debug-wk2
	❌ 🛠 watch	❌ 🧪 mac-wk2-stress
	❌ 🛠 watch-sim

webkit-early-warning-system · 2023-02-13T20:43:07Z

EWS run on previous version of this PR (hash a9ad486)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 gtk-wk2
	❌ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🧪 api-gtk
	❌ 🛠 tv	❌ 🧪 mac-wk2
	❌ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2
	❌ 🛠 watch	✅ 🧪 mac-wk2-stress
	❌ 🛠 watch-sim

webkit-early-warning-system · 2023-02-15T21:01:19Z

EWS run on previous version of this PR (hash 2272cbd)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	✅ 🧪 api-mac	❌ 🧪 gtk-wk2
	❌ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🧪 api-gtk
	❌ 🛠 tv	✅ 🧪 mac-wk2
	❌ 🛠 tv-sim	❌ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch	✅ 🧪 mac-wk2-stress
	✅ 🛠 watch-sim

Source/WebCore/html/HTMLMediaElement.h

webkit-early-warning-system · 2023-02-17T00:03:59Z

EWS run on previous version of this PR (hash f8624ec)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	❌ 🛠 ios	❌ 🛠 mac	✅ 🛠 wpe	⏳ 🛠 wincairo
✅ 🧪 bindings	❌ 🛠 ios-sim	❌ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	❌ 🧪 api-mac	~~🧪 gtk-wk2~~
	❌ 🧪 api-ios	❌ 🧪 mac-wk1	~~🧪 api-gtk~~
⏳ 🛠 🧪 jsc	✅ 🛠 tv	❌ 🧪 mac-wk2	⏳ 🛠 jsc-armv7
	✅ 🛠 tv-sim	❌ 🧪 mac-AS-debug-wk2	⏳ 🧪 jsc-armv7-tests
	❌ 🛠 watch	❌ 🧪 mac-wk2-stress
	❌ 🛠 watch-sim

webkit-early-warning-system · 2023-02-17T00:04:00Z

EWS run on previous version of this PR (hash d073f0c)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
⏳ 🧪 style	❌ 🛠 ios	⏳ 🛠 mac	⏳ 🛠 wpe	⏳ 🛠 wincairo
⏳ 🧪 bindings	⏳ 🛠 ios-sim	⏳ 🛠 mac-AS-debug	❌ 🛠 gtk
⏳ 🧪 webkitperl	⏳ 🧪 ios-wk2	⏳ 🧪 api-mac	❌ 🧪 gtk-wk2
	⏳ 🧪 api-ios	⏳ 🧪 mac-wk1	❌ 🧪 api-gtk
⏳ 🛠 🧪 jsc	⏳ 🛠 tv	⏳ 🧪 mac-wk2	⏳ 🛠 jsc-armv7
⏳ 🛠 🧪 jsc-arm64	~~🛠 tv-sim~~	⏳ 🧪 mac-AS-debug-wk2	⏳ 🧪 jsc-armv7-tests
⏳ 🧪 services	⏳ 🛠 watch	⏳ 🧪 mac-wk2-stress	⏳ 🛠 jsc-mips
	❌ 🛠 watch-sim		⏳ 🧪 jsc-mips-tests

webkit-early-warning-system · 2023-02-17T02:07:14Z

EWS run on previous version of this PR (hash da3eaa7)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	~~🛠 ios-sim~~	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	~~🧪 ios-wk2~~	✅ 🧪 api-mac	✅ 🧪 gtk-wk2
	❌ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch	✅ 🧪 mac-wk2-stress
	✅ 🛠 watch-sim

pvollan · 2023-02-17T16:03:09Z

Source/WebKit/Shared/WebExtensionContextIdentifier.h

This change looks somewhat unrelated, is it strictly needed?

pvollan · 2023-02-17T16:22:06Z

Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp

There's missing a new line between these methods.

pvollan · 2023-02-17T17:07:39Z

Source/WebCore/platform/graphics/GraphicsLayer.h

In order to avoid referencing HTML element classes inside the graphics layer class, you could possibly create a new, more generic class to use here instead of HTMLVideoElement. This class would then declare the virtual methods needed, and HTMLVideElement could subclass and override the methods. However, I don't think this is needed to move forward with the current patch, and could be done in a follow-up patch, if you think it makes sense.

GraphicsLayer cannot know about HMTL elements.

pvollan

R=me.

webkit-early-warning-system · 2023-02-17T20:25:21Z

EWS run on current version of this PR (hash 89dd664)

Details

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 wincairo
✅ 🧪 bindings	⏳ 🛠 ios-sim	✅ 🛠 mac-AS-debug	✅ 🛠 gtk
✅ 🧪 webkitperl	⏳ 🧪 ios-wk2	✅ 🧪 api-mac	✅ 🧪 gtk-wk2
	⏳ 🧪 api-ios	✅ 🧪 mac-wk1	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-wk2
	✅ 🛠 tv-sim	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch	✅ 🧪 mac-wk2-stress
✅ 🛠 🧪 merge	✅ 🛠 watch-sim

…ing is enabled https://bugs.webkit.org/show_bug.cgi?id=252021 <rdar://105246033> Reviewed by Eric Carlson and Per Arne Vollan. Currently, video content from the GPU process is remotely-hosted into the WebContent process, and is subsequently remotely hosted from the WebContent process to the UI process when UI-side compositing is enabled. This is unnecessary, and necessitates having an active CARenderer in the Web Content process. Rather than double hosting video content from GPU -> WebContent -> UI process, video content should be hosted directly from the GPU process into the UI process. To facilitate this, re-use the infrastructure built into VideoFullscreenManager to handle the required XPC communication to send resize information back down from the UI process to the GPU process. HTMLMediaElement will need to expose API to resize its video layer, and to provide the remote hosting context ID from the GPU process. Rather than have a separate identifier of its own, VideoFullscreenManager and PlaybackSessionManager can just use a HTMLMediaElement identifier for the purposes of passing messages across the XPC boundary. WebAVPlayerLayer must support use cases more like VideoLayerRemoteCocoa, including handling multiple VideoGravity settings, so it is improved to take these settings into account. WebAVPlayerLayerView has been pulled out of VideoFullScreenManagerAVKit into its own source and header files. GraphicsLayerCA, RemoteLayerTreeHost, and RemoteLayerTree all must be modified to accept a HTMLVideoElement (or its identifier) as a parameter. * Source/WebCore/WebCore.xcodeproj/project.pbxproj: * Source/WebCore/dom/ImageOverlay.cpp: (WebCore::ImageOverlay::updateSubtree): * Source/WebCore/html/HTMLMediaElement.cpp: (WebCore::HTMLMediaElement::setVideoInlineSizeFenced): (WebCore::HTMLMediaElement::layerHostingContextID): (WebCore::HTMLMediaElement::naturalSize): * Source/WebCore/html/HTMLMediaElement.h: (WebCore::HTMLMediaElement::identifier const): * Source/WebCore/platform/cocoa/VideoFullscreenModel.h: * Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.h: * Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.mm: (WebCore::VideoFullscreenModelVideoElement::setVideoInlineSizeFenced): * Source/WebCore/platform/cocoa/WebAVPlayerLayer.h: * Source/WebCore/platform/cocoa/WebAVPlayerLayer.mm: (-[WebAVPlayerLayer layoutSublayers]): (-[WebAVPlayerLayer resolveBounds]): * Source/WebCore/platform/graphics/GraphicsLayer.h: (WebCore::GraphicsLayer::layerMode const): (WebCore::GraphicsLayer::setContentsToVideoElement): * Source/WebCore/platform/graphics/MediaPlayer.cpp: (WebCore::MediaPlayer::setVideoInlineSizeFenced): (WebCore::MediaPlayer::hostingContextID const): * Source/WebCore/platform/graphics/MediaPlayer.h: * Source/WebCore/platform/graphics/MediaPlayerPrivate.h: (WebCore::MediaPlayerPrivateInterface::setVideoInlineSizeFenced): (WebCore::MediaPlayerPrivateInterface::hostingContextID const): * Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp: (WebCore::GraphicsLayerCA::createPlatformCALayer): (WebCore::GraphicsLayerCA::setContentsToPlatformLayer): (WebCore::GraphicsLayerCA::setContentsToVideoElement): * Source/WebCore/platform/graphics/ca/GraphicsLayerCA.h: * Source/WebCore/platform/graphics/ca/PlatformCALayer.cpp: (WebCore::operator<<): * Source/WebCore/platform/graphics/ca/PlatformCALayer.h: * Source/WebCore/platform/graphics/ca/cocoa/PlatformCALayerCocoa.mm: (WebCore::PlatformCALayerCocoa::PlatformCALayerCocoa): * Source/WebCore/platform/graphics/ca/cocoa/WebVideoContainerLayer.mm: (-[WebVideoContainerLayer setPosition:]): Deleted. * Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h: * Source/WebCore/platform/ios/VideoFullscreenInterfaceAVKit.h: * Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.h: * Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.mm: (-[WebVideoFullscreenInterfaceMacObjC setUpPIPForVideoView:withFrame:inWindow:]): (WebCore::VideoFullscreenInterfaceMac::createLayer): * Source/WebCore/rendering/RenderLayerBacking.cpp: (WebCore::RenderLayerBacking::updateConfiguration): * Source/WebKit/Platform/cocoa/LayerHostingContext.h: * Source/WebKit/Platform/cocoa/LayerHostingContext.mm: (WebKit::LayerHostingContext::createForPort): (WebKit::LayerHostingContext::createForExternalHostingProcess): (WebKit::LayerHostingContext::createTransportLayerForRemoteHosting): (WebKit::LayerHostingContext::updateCachedContextID): (WebKit::LayerHostingContext::cachedContextID): * Source/WebKit/Shared/PlaybackSessionContextIdentifier.h: (): Deleted. * Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm: (WebKit::RemoteLayerBackingStore::drawInContext): * Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h: * Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm: (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties): (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode const): (WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode): (WebKit::RemoteLayerTreeTransaction::description const): * Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.h: * Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.mm: (WebKit::VideoFullscreenManagerProxy::createLayerWithID): (WebKit::VideoFullscreenManagerProxy::createLayerHostViewWithID): (WebKit::VideoFullscreenManagerProxy::setupFullscreenWithID): (WebKit::VideoFullscreenManagerProxy::didCleanupFullscreen): * Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm: (WebKit::RemoteLayerTreeHost::makeNode): * Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm: (WebKit::RemoteLayerTreeHost::makeNode): * Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp: (WebKit::MediaPlayerPrivateRemote::hostingContextID const): (WebKit::MediaPlayerPrivateRemote::setLayerHostingContextID): (WebKit::MediaPlayerPrivateRemote::setCachedPresentationSize): (WebKit::MediaPlayerPrivateRemote::cachedPresentationSize): * Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h: * Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm: (WebKit::MediaPlayerPrivateRemote::layerHostingContextIdChanged): * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp: (WebKit::GraphicsLayerCARemote::createPlatformCALayer): (WebKit::GraphicsLayerCARemote::layerMode const): * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h: * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp: (WebKit::PlatformCALayerRemote::create): (WebKit::PlatformCALayerRemote::recursiveBuildTransaction): * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h: * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.h: * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.mm: (WebKit::PlatformCALayerRemoteCustom::create): (WebKit::PlatformCALayerRemoteCustom::PlatformCALayerRemoteCustom): (WebKit::PlatformCALayerRemoteCustom::hostingContextID): * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.h: (WebKit::RemoteLayerTreeContext::webPage): * Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm: (WebKit::RemoteLayerTreeContext::layerDidEnterContext): * Source/WebKit/WebProcess/WebPage/WebPage.cpp: (WebKit::m_appHighlightsVisible): * Source/WebKit/WebProcess/WebPage/mac/TiledCoreAnimationDrawingArea.mm: (WebKit::TiledCoreAnimationDrawingArea::sendDidFirstLayerFlushIfNeeded): (WebKit::TiledCoreAnimationDrawingArea::sendEnterAcceleratedCompositingModeIfNeeded): (WebKit::TiledCoreAnimationDrawingArea::setLayerHostingMode): * Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h: * Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm: (WebKit::PlaybackSessionManager::removeContext): (WebKit::PlaybackSessionManager::setUpPlaybackControlsManager): (WebKit::PlaybackSessionManager::contextIdForMediaElement): * Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.h: * Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.mm: (WebKit::VideoFullscreenManager::setupRemoteLayerHosting): (WebKit::VideoFullscreenManager::enterVideoFullscreenForVideoElement): (WebKit::VideoFullscreenManager::setVideoLayerFrameFenced): Co-authored-by: Arunsundar Kannan <arunsundar_kannan@apple.com> Canonical link: https://commits.webkit.org/260575@main

webkit-commit-queue · 2023-02-21T00:12:32Z

Committed 260575@main (897c865): https://commits.webkit.org/260575@main

Reviewed commits have been landed. Closing PR #9910 and removing active labels.

smfr · 2023-02-21T00:35:40Z

Source/WebCore/platform/graphics/GraphicsLayer.h

 #include "FloatRoundedRect.h"
 #include "FloatSize.h"
 #include "GraphicsLayerClient.h"
+#include "HTMLMediaElementIdentifier.h"


This is a layering violation.

smfr · 2023-02-21T00:36:02Z

Source/WebCore/platform/graphics/GraphicsLayer.h

GraphicsLayer cannot know about HMTL elements.

smfr · 2023-02-21T00:36:49Z

Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp

    noteLayerPropertyChanged(ContentsPlatformLayerChanged);
 }

+void GraphicsLayerCA::setContentsToVideoElement(HTMLVideoElement& videoElement, ContentsLayerPurpose purpose)


The way to fix the layering violation would be to call something on the HTMLVideoElement that sets up this GraphicsLayer.

jernoble requested review from cdumez and rniwa as code owners February 10, 2023 08:25

jernoble self-assigned this Feb 10, 2023

jernoble added the Media Bugs related to the HTML 5 Media elements. label Feb 10, 2023

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from 554a0ba to 3416692 Compare February 10, 2023 17:08

eric-carlson approved these changes Feb 10, 2023

View reviewed changes

pvollan self-requested a review February 10, 2023 20:23

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from 3416692 to f30834a Compare February 10, 2023 22:41

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from f30834a to a9ad486 Compare February 13, 2023 20:41

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from a9ad486 to 2272cbd Compare February 15, 2023 20:53

jernoble requested review from b-weinstein and xeenon as code owners February 15, 2023 20:53

pvollan reviewed Feb 15, 2023

View reviewed changes

Source/WebCore/html/HTMLMediaElement.h Outdated Show resolved Hide resolved

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch 2 times, most recently from d073f0c to f8624ec Compare February 16, 2023 23:56

jernoble requested a review from pvollan February 16, 2023 23:56

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from f8624ec to da3eaa7 Compare February 17, 2023 01:45

pvollan reviewed Feb 17, 2023

View reviewed changes

pvollan approved these changes Feb 17, 2023

View reviewed changes

jernoble force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from da3eaa7 to 89dd664 Compare February 17, 2023 20:04

jernoble added the merge-queue Applied to send a pull request to merge-queue label Feb 20, 2023

webkit-early-warning-system force-pushed the eng/Eliminate-double-layer-hosting-of-video-content-when-UI-Side-compositing-is-enabled branch from 89dd664 to 897c865 Compare February 21, 2023 00:12

webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Feb 21, 2023

webkit-early-warning-system merged commit 897c865 into WebKit:main Feb 21, 2023

smfr reviewed Feb 21, 2023

View reviewed changes

Eliminate double layer hosting of video content when UI-Side compositing is enabled #9910

Eliminate double layer hosting of video content when UI-Side compositing is enabled #9910

Uh oh!

Conversation

jernoble commented Feb 10, 2023 • edited by webkit-early-warning-system Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eric-carlson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

webkit-early-warning-system commented Feb 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-early-warning-system commented Feb 17, 2023

Uh oh!

webkit-early-warning-system commented Feb 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pvollan left a comment

Choose a reason for hiding this comment

Uh oh!

webkit-early-warning-system commented Feb 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

webkit-commit-queue commented Feb 21, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jernoble commented Feb 10, 2023 •

edited by webkit-early-warning-system

Loading

webkit-early-warning-system commented Feb 10, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 10, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 10, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 13, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 15, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 17, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 17, 2023 •

edited

Loading

webkit-early-warning-system commented Feb 17, 2023 •

edited

Loading