border-radius clipping of composited layers doesn't work

[smfr]

9791054

border-radius clipping of composited layers doesn't work
https://bugs.webkit.org/show_bug.cgi?id=68196
<rdar://10133719>

Reviewed by Alan Bujtas.

In CSS it's possible for clipping (via overflow:scroll, overflow:hidden and the `clip`
property) to apply to elements that are descendants in containing-block order, but
siblings in z-order, for example:
    <div style="position: relative; overflow: hidden">
      <div style="position: absolute"></div
    </div>
If a non-z-order descendant here has a composited layer, we'd fail to take border-radius
into account when clipping it.

In this layer configuration we fall into the "AncestorClippingStack" code path: the
descendant layer owns a stack of clips that represent clipping by non-paint-order
ancestors. Previously, all overflow:hidden clips were simply intersected and represented
as a single rectangular clipping GraphicsLayer, but when border-radius is present on
some or all of them, we need to maintain a layer for each, since you can't trivially
generate a path that is the intersection of rounded rects. Code was added to
RenderLayerCompositor::computeAncestorClippingStack() to do this.

When an intermediate clip with border radius represents a scrollable elements (i.e.
scrollable overflow), its corresponding entry in the ancestor clipping stack needs two
layers; one to apply the clipping, which doesn't scroll with the content, and one that
applies the scroll offset, so ClippingStackEntry gains a scrollingLayer, which is the
one handed off to the scrolling tree, and the rounded clip is applied to the clipping
layer. Logic is added to build the hierarchy of these layers.

When style changes we have to ensure that layers are updated with the new rounded rects;
notably, style can change on a non-composited RenderLayer (e.g. an overflow:hidden layer
with border-radius) which requires that non-z-order descendants need to update their
composited layer geometry; new code in RenderLayerCompositor::layerStyleChanged() takes
care of this. Tested by
compositing/clipping/border-radius-with-composited-descendant-dynamic.html

* LayoutTests/compositing/clipping/border-radius-async-overflow-non-stacking.html:
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant-dynamic-expected.html: Added.
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant-dynamic.html: Added.
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant-expected.html: Added.
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant-nested-expected.html: Added.
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant-nested.html: Added.
* LayoutTests/compositing/clipping/border-radius-with-composited-descendant.html: Added.
* LayoutTests/compositing/layer-creation/clipping-scope/nested-scroller-overlap-expected.txt:
* LayoutTests/compositing/layer-creation/clipping-scope/overlap-constrained-inside-scroller-expected.txt:
* LayoutTests/compositing/layer-creation/clipping-scope/scroller-with-negative-z-children-expected.txt:
* LayoutTests/compositing/overflow/scrolling-content-clip-to-viewport-expected.txt:
* LayoutTests/compositing/rtl/rtl-scrolling-with-transformed-descendants-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-clipped-by-scroll-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-nested-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-for-negative-z-in-scroller-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-clip-to-hidden-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-clip-to-visible-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-gain-clipping-layer-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-in-clipped-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-lose-clipping-layer-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/nested-scrollers-backing-attachment-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/overlapped-overlay-scrollbar-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/overlapped-overlay-scrollbar-inside-hidden-expected.txt:
* LayoutTests/compositing/scrolling/async-overflow-scrolling/overlapped-overlay-scrollbar-nested-expected.txt:
* LayoutTests/compositing/shared-backing/overflow-scroll/composited-absolute-in-absolute-in-relative-in-scroller-expected.txt:
* LayoutTests/compositing/shared-backing/overflow-scroll/previous-sibling-prevents-inclusiveness-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/layer-creation/clipping-scope/nested-scroller-overlap-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/layer-creation/clipping-scope/overlap-constrained-inside-scroller-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/layer-creation/clipping-scope/scroller-with-negative-z-children-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-clipped-by-scroll-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/clipped-layer-in-overflow-nested-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-for-negative-z-in-scroller-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-clip-to-hidden-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-clip-to-visible-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-gain-clipping-layer-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-in-clipped-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/layer-in-overflow-lose-clipping-layer-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/scrolling/async-overflow-scrolling/nested-scrollers-backing-attachment-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/shared-backing/overflow-scroll/composited-absolute-in-absolute-in-relative-in-scroller-expected.txt:
* LayoutTests/platform/ios-wk2/compositing/shared-backing/overflow-scroll/previous-sibling-prevents-inclusiveness-expected.txt:
* Source/WebCore/platform/graphics/RoundedRect.cpp:
(WebCore::operator<<):
* Source/WebCore/platform/graphics/RoundedRect.h:
* Source/WebCore/rendering/LayerAncestorClippingStack.cpp:
(WebCore::LayerAncestorClippingStack::LayerAncestorClippingStack):
(WebCore::LayerAncestorClippingStack::firstLayer const):
(WebCore::LayerAncestorClippingStack::lastLayer const):
(WebCore::LayerAncestorClippingStack::updateScrollingNodeLayers):
(WebCore::LayerAncestorClippingStack::updateWithClipData):
(WebCore::operator<<):
(WebCore::LayerAncestorClippingStack::firstClippingLayer const): Deleted.
(WebCore::LayerAncestorClippingStack::lastClippingLayer const): Deleted.
* Source/WebCore/rendering/LayerAncestorClippingStack.h:
(WebCore::CompositedClipData::CompositedClipData):
(WebCore::LayerAncestorClippingStack::ClippingStackEntry::parentForSublayers const):
(WebCore::LayerAncestorClippingStack::ClippingStackEntry::childForSuperlayers const):
* Source/WebCore/rendering/RenderLayer.h:
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateInternalHierarchy):
(WebCore::RenderLayerBacking::ensureClippingStackLayers):
(WebCore::RenderLayerBacking::removeClippingStackLayers):
(WebCore::RenderLayerBacking::connectClippingStackLayers):
(WebCore::RenderLayerBacking::updateClippingStackLayerGeometry):
(WebCore::RenderLayerBacking::childForSuperlayers const):
* Source/WebCore/rendering/RenderLayerCompositor.cpp:
(WebCore::RenderLayerCompositor::adjustOverflowScrollbarContainerLayers):
(WebCore::RenderLayerCompositor::layerStyleChanged):
(WebCore::RenderLayerCompositor::computeAncestorClippingStack const):
(WebCore::RenderLayerCompositor::parentRelativeScrollableRect const):
(WebCore::RenderLayerCompositor::updateScrollingNodeForScrollingProxyRole):
* Source/WebCore/rendering/RenderLayerCompositor.h:

Canonical link: https://commits.webkit.org/254253@main

ee7713b

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 🧪 win
	✅ 🛠 ios-sim	✅ 🛠 mac-debug	✅ 🛠 gtk	✅ 🛠 wincairo
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🛠 mac-AS-debug	✅ 🧪 gtk-wk2
	✅ 🧪 api-ios	🧪 api-mac	✅ 🧪 api-gtk
	✅ 🛠 tv	🧪 mac-wk1
	🛠 tv-sim	✅ 🧪 mac-wk2
✅ 🛠 🧪 merge	✅ 🛠 watch	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch-sim	✅ 🧪 mac-wk2-stress

[webkit-early-warning-system]

EWS run on previous version of this PR (hash 2aa3a45)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
❌ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 🧪 win
✅ 🧪 bindings	✅ 🛠 ios-sim	✅ 🛠 mac-debug	✅ 🛠 gtk	✅ 🛠 wincairo
✅ 🧪 webkitperl	❌ 🧪 ios-wk2	✅ 🛠 mac-AS-debug	✅ 🧪 gtk-wk2
	✅ 🧪 api-ios	✅ 🧪 api-mac	✅ 🧪 api-gtk
	✅ 🛠 tv	✅ 🧪 mac-wk1
	✅ 🛠 tv-sim	❌ 🧪 mac-wk2
	✅ 🛠 watch	❌ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch-sim	✅ 🧪 mac-wk2-stress

[webkit-early-warning-system]

EWS run on previous version of this PR (hash ec76312)

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 🧪 win
	✅ 🛠 ios-sim	✅ 🛠 mac-debug	✅ 🛠 gtk	✅ 🛠 wincairo
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🛠 mac-AS-debug	✅ 🧪 gtk-wk2
	✅ 🧪 api-ios	✅ 🧪 api-mac	✅ 🧪 api-gtk
	✅ 🛠 tv	🧪 mac-wk1
	✅ 🛠 tv-sim	✅ 🧪 mac-wk2
	✅ 🛠 watch	❌ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch-sim	✅ 🧪 mac-wk2-stress

[webkit-early-warning-system]

ee7713b

Misc	iOS, tvOS & watchOS	macOS	Linux	Windows
✅ 🧪 style	✅ 🛠 ios	✅ 🛠 mac	✅ 🛠 wpe	✅ 🛠 🧪 win
	✅ 🛠 ios-sim	✅ 🛠 mac-debug	✅ 🛠 gtk	✅ 🛠 wincairo
✅ 🧪 webkitperl	✅ 🧪 ios-wk2	✅ 🛠 mac-AS-debug	✅ 🧪 gtk-wk2
	✅ 🧪 api-ios	🧪 api-mac	✅ 🧪 api-gtk
	✅ 🛠 tv	🧪 mac-wk1
	🛠 tv-sim	✅ 🧪 mac-wk2
✅ 🛠 🧪 merge	✅ 🛠 watch	✅ 🧪 mac-AS-debug-wk2
	✅ 🛠 watch-sim	✅ 🧪 mac-wk2-stress

[webkit-commit-queue]

Committed 254253@main (9791054): https://commits.webkit.org/254253@main

Reviewed commits have been landed. Closing PR #3883 and removing active labels.