-
Notifications
You must be signed in to change notification settings - Fork 3.4k
test_driver.click() is unreliable because of coordinate rounding issues #53241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…es (web-platform-tests#53239) `test_driver.click()` determines if an element is targetable by computing its client rect, truncated to the viewport, and calling `elementsFromPoint()` at the center of the rect. If that finds the element, then it dispatches an event using those center coordinates. The problem is that client rects, and `elementsFromPoint()`, support fractional x and y values, but mouse events do not (x and y get floored). The code can thus think that an element is in view, but a dispatched event fails to hit it. In the example I investigated, the target element's top was 599.5px in 600px tall viewport, so the center point was y=599.75 and elementsFromPoint found the element, but the event dispatch at y=599 failed to hit it. Fix by ensuring that at least 1px of the element is visible by adding `rectConstrainedToViewport()`. Use 'const' and 'let' in this code. Rename the confusing `getPointerInteractablePaintTree` to `getElementListAtVisibleCenter`.
https://bugs.webkit.org/show_bug.cgi?id=294571 rdar://152114545 Reviewed by NOBODY (OOPS!). IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically painted and hit-tested using integer coordinates. However, iframes are often positioned on subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI. However, we can adjust iframes for painting and hit-testing to overcome the integral Widget limitation. This PR does this in various places. First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position, which fixes painting gaps (tested by iframe-subpixel-painting.html). Second, the compositing code needs to handle subpixel positioning of the iframe's layers, fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by iframe-subpixel-compositing.html. Finally, hit-testing needs to maintain fractional coordinates; `documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the `nodeAtPoint()` code. Removing the flooring of the point in the HitTestLocation constructor caused `css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an element with a 100x scale. The existing code would map the 1x1px hit-test area through the scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to maintain the 1x1px hit-test area in screen space. This is a merge of https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d * LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html fails; web-platform-tests/wpt#53241 exists to fix this. * LayoutTests/compositing/iframes/iframe-subpixel-compositing-expected.html: Added. * LayoutTests/compositing/iframes/iframe-subpixel-compositing.html: Added. * LayoutTests/fast/frames/iframe-subpixel-hit-test-expected.txt: Added. * LayoutTests/fast/frames/iframe-subpixel-hit-test.html: Added. * LayoutTests/fast/frames/iframe-subpixel-painting-expected.html: Added. * LayoutTests/fast/frames/iframe-subpixel-painting.html: Added. * LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers have different results. * Source/WebCore/page/EventHandler.cpp: (WebCore::documentPointForWindowPoint): * Source/WebCore/page/scrolling/ScrollingCoordinator.cpp: (WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const): * Source/WebCore/rendering/HitTestLocation.cpp: (WebCore::HitTestLocation::HitTestLocation): No longer floor the point. (WebCore::HitTestLocation::move): No longer convert to IntRect. * Source/WebCore/rendering/HitTestingTransformState.cpp: (WebCore::HitTestingTransformState::boundsOfMappedQuad const): * Source/WebCore/rendering/HitTestingTransformState.h: * Source/WebCore/rendering/RenderLayer.cpp: (WebCore::RenderLayer::hitTestLayerByApplyingTransform): * Source/WebCore/rendering/RenderLayerBacking.cpp: (WebCore::RenderLayerBacking::updateAfterWidgetResize): * Source/WebCore/rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::frameViewDidChangeLocation): * Source/WebCore/rendering/RenderLayerCompositor.h: * Source/WebCore/rendering/RenderWidget.cpp: (WebCore::RenderWidget::paintContents):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comments below; also cc @masayuki-nakano, @mustaqahmed, @smaug----.
We probably need to make sure this doesn't affect any of the Pointer Events or UI Events tests, at least.
const rectInView = rectConstrainedToViewport(clientRect); | ||
|
||
// Ensure that at least 1px is visible to avoid issues due to elementsFromPoint() | ||
// supporting fractional values, but mouse events flooring to integers. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, them flooring to integers depends on which spec you're looking at, given https://w3c.github.io/uievents/#mouseevent and https://drafts.csswg.org/cssom-view/#extensions-to-the-mouseevent-interface give differing definitions for the same attributes. (See w3c/uievents#24 & w3c/csswg-drafts#4084.)
https://w3c.github.io/pointerevents/#event-coordinates also says "user agents that have implemented the proposed change in CSSOM View Module only for PointerEvent", which implies this doesn't apply to all user agents.
I'm mostly slightly weary about making it impossible to test the Pointer Events behaviour here by introducing rounding always.
Something similar to what we do in this pointer events test might work, checking that at least 1 pixel is in view when fractional values aren't supported, and checking greater than 0 pixels are in view when they are:
if ((new PointerEvent("click", {screenX: 0.1})).screenX == 0.1) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://w3c.github.io/pointerevents/#event-coordinates also says "user agents that have implemented the proposed change in CSSOM View Module only for PointerEvent", which implies this doesn't apply to all user agents.
If fractional coordinates in click a concern here (I am not sure), we do not need to support that to the best of my knowledge:
- IIRC, "user agents that have implemented..." was added to cover the unclear "requirement" around CSSOM-View extension: fractional coordinates are neither required not optional because of Converge with CSSOM-View w3c/uievents#24!
- We (Blink) already confirmed that non-integer coordinates are not web compatible for
click
-like events.
return new DOMRect(rect.x, rect.y, Math.max(right - left, 0), Math.max(bottom - top, 0)); | ||
} | ||
|
||
function getElementListAtVisibleCenter(element) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function was intended to match https://w3c.github.io/webdriver/#dfn-pointer-interactable-paint-tree, and named as such.
I'm not saying we can't deviate from WebDriver here, but it does hint at there maybe being issues in the WebDriver definitions here.
https://bugs.webkit.org/show_bug.cgi?id=294571 rdar://152114545 Reviewed by NOBODY (OOPS!). IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically painted and hit-tested using integer coordinates. However, iframes are often positioned on subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI. However, we can adjust iframes for painting and hit-testing to overcome the integral Widget limitation. This PR does this in various places. First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position, which fixes painting gaps (tested by iframe-subpixel-painting.html). Second, the compositing code needs to handle subpixel positioning of the iframe's layers, fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by iframe-subpixel-compositing.html. Finally, hit-testing needs to maintain fractional coordinates; `documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the `nodeAtPoint()` code. Removing the flooring of the point in the HitTestLocation constructor caused `css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an element with a 100x scale. The existing code would map the 1x1px hit-test area through the scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to maintain the 1x1px hit-test area in screen space. This is a merge of https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d * LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html fails; web-platform-tests/wpt#53241 exists to fix this. * LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing-expected.html: Added. * LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test-expected.txt: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-painting-expected.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-painting.html: Added. * LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers have different results. * Source/WebCore/page/EventHandler.cpp: (WebCore::documentPointForWindowPoint): * Source/WebCore/page/scrolling/ScrollingCoordinator.cpp: (WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const): * Source/WebCore/rendering/HitTestLocation.cpp: (WebCore::HitTestLocation::HitTestLocation): No longer floor the point. (WebCore::HitTestLocation::move): No longer convert to IntRect. * Source/WebCore/rendering/HitTestingTransformState.cpp: (WebCore::HitTestingTransformState::boundsOfMappedQuad const): * Source/WebCore/rendering/HitTestingTransformState.h: * Source/WebCore/rendering/RenderLayer.cpp: (WebCore::RenderLayer::hitTestLayerByApplyingTransform): * Source/WebCore/rendering/RenderLayerBacking.cpp: (WebCore::RenderLayerBacking::updateAfterWidgetResize): * Source/WebCore/rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::frameViewDidChangeLocation): * Source/WebCore/rendering/RenderLayerCompositor.h: * Source/WebCore/rendering/RenderWidget.cpp: (WebCore::RenderWidget::paintContents):
https://bugs.webkit.org/show_bug.cgi?id=294571 rdar://152114545 Reviewed by Alan Baradlay. IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically painted and hit-tested using integer coordinates. However, iframes are often positioned on subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI. However, we can adjust iframes for painting and hit-testing to overcome the integral Widget limitation. This PR does this in various places. First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position, which fixes painting gaps (tested by iframe-subpixel-painting.html). Second, the compositing code needs to handle subpixel positioning of the iframe's layers, fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by iframe-subpixel-compositing.html. Finally, hit-testing needs to maintain fractional coordinates; `documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the `nodeAtPoint()` code. Removing the flooring of the point in the HitTestLocation constructor caused `css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an element with a 100x scale. The existing code would map the 1x1px hit-test area through the scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to maintain the 1x1px hit-test area in screen space. This is a merge of https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d * LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html fails; web-platform-tests/wpt#53241 exists to fix this. * LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing-expected.html: Added. * LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test-expected.txt: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-painting-expected.html: Added. * LayoutTests/fast/frames/hidpi-iframe-subpixel-painting.html: Added. * LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt: * LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers have different results. * Source/WebCore/page/EventHandler.cpp: (WebCore::documentPointForWindowPoint): * Source/WebCore/page/scrolling/ScrollingCoordinator.cpp: (WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const): * Source/WebCore/rendering/HitTestLocation.cpp: (WebCore::HitTestLocation::HitTestLocation): No longer floor the point. (WebCore::HitTestLocation::move): No longer convert to IntRect. * Source/WebCore/rendering/HitTestingTransformState.cpp: (WebCore::HitTestingTransformState::boundsOfMappedQuad const): * Source/WebCore/rendering/HitTestingTransformState.h: * Source/WebCore/rendering/RenderLayer.cpp: (WebCore::RenderLayer::hitTestLayerByApplyingTransform): * Source/WebCore/rendering/RenderLayerBacking.cpp: (WebCore::RenderLayerBacking::updateAfterWidgetResize): * Source/WebCore/rendering/RenderLayerCompositor.cpp: (WebCore::RenderLayerCompositor::frameViewDidChangeLocation): * Source/WebCore/rendering/RenderLayerCompositor.h: * Source/WebCore/rendering/RenderWidget.cpp: (WebCore::RenderWidget::paintContents): Canonical link: https://commits.webkit.org/296407@main
test_driver.click()
determines if an element is targetable by computing its client rect, truncated to the viewport, and callingelementsFromPoint()
at the center of the rect. If that finds the element, then it dispatches an event using those center coordinates.The problem is that client rects, and
elementsFromPoint()
, support fractional x and y values, but mouse events do not (x and y get floored). The code can thus think that an element is in view, but a dispatched event fails to hit it.In the example I investigated, the target element's top was 599.5px in 600px tall viewport, so the center point was y=599.75 and elementsFromPoint found the element, but the event dispatch at y=599 failed to hit it.
Fix by ensuring that at least 1px of the element is visible by adding
rectConstrainedToViewport()
.Use 'const' and 'let' in this code.
Rename the confusing
getPointerInteractablePaintTree
togetElementListAtVisibleCenter
.Fixes #53239