Skip to content

test_driver.click() is unreliable because of coordinate rounding issues #53241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

smfr
Copy link
Contributor

@smfr smfr commented Jun 18, 2025

test_driver.click() determines if an element is targetable by computing its client rect, truncated to the viewport, and calling elementsFromPoint() at the center of the rect. If that finds the element, then it dispatches an event using those center coordinates.

The problem is that client rects, and elementsFromPoint(), support fractional x and y values, but mouse events do not (x and y get floored). The code can thus think that an element is in view, but a dispatched event fails to hit it.

In the example I investigated, the target element's top was 599.5px in 600px tall viewport, so the center point was y=599.75 and elementsFromPoint found the element, but the event dispatch at y=599 failed to hit it.

Fix by ensuring that at least 1px of the element is visible by adding rectConstrainedToViewport().

Use 'const' and 'let' in this code.

Rename the confusing getPointerInteractablePaintTree to getElementListAtVisibleCenter.

Fixes #53239

…es (web-platform-tests#53239)

`test_driver.click()` determines if an element is targetable by computing its
client rect, truncated to the viewport, and calling `elementsFromPoint()` at the
center of the rect. If that finds the element, then it dispatches an event using
those center coordinates.

The problem is that client rects, and `elementsFromPoint()`, support fractional
x and y values, but mouse events do not (x and y get floored). The code can thus
think that an element is in view, but a dispatched event fails to hit it.

In the example I investigated, the target element's top was 599.5px in 600px tall
viewport, so the center point was y=599.75 and elementsFromPoint found the element,
but the event dispatch at y=599 failed to hit it.

Fix by ensuring that at least 1px of the element is visible by adding
`rectConstrainedToViewport()`.

Use 'const' and 'let' in this code.

Rename the confusing `getPointerInteractablePaintTree` to
`getElementListAtVisibleCenter`.
@smfr smfr requested a review from a team as a code owner June 18, 2025 01:42
@wpt-pr-bot wpt-pr-bot requested a review from jgraham June 18, 2025 01:42
smfr added a commit to smfr/WebKit that referenced this pull request Jun 18, 2025
https://bugs.webkit.org/show_bug.cgi?id=294571
rdar://152114545

Reviewed by NOBODY (OOPS!).

IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically
painted and hit-tested using integer coordinates. However, iframes are often positioned on
subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an
iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI.

However, we can adjust iframes for painting and hit-testing to overcome the integral
Widget limitation. This PR does this in various places.

First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position,
which fixes painting gaps (tested by iframe-subpixel-painting.html).

Second, the compositing code needs to handle subpixel positioning of the iframe's layers,
fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by
iframe-subpixel-compositing.html.

Finally, hit-testing needs to maintain fractional coordinates;
`documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and
HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the
`nodeAtPoint()` code.

Removing the flooring of the point in the HitTestLocation constructor caused
`css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an
element with a 100x scale. The existing code would map the 1x1px hit-test area through the
scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to
maintain the 1x1px hit-test area in screen space. This is a merge of
https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d

* LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html
fails; web-platform-tests/wpt#53241 exists to fix this.
* LayoutTests/compositing/iframes/iframe-subpixel-compositing-expected.html: Added.
* LayoutTests/compositing/iframes/iframe-subpixel-compositing.html: Added.
* LayoutTests/fast/frames/iframe-subpixel-hit-test-expected.txt: Added.
* LayoutTests/fast/frames/iframe-subpixel-hit-test.html: Added.
* LayoutTests/fast/frames/iframe-subpixel-painting-expected.html: Added.
* LayoutTests/fast/frames/iframe-subpixel-painting.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers
have different results.
* Source/WebCore/page/EventHandler.cpp:
(WebCore::documentPointForWindowPoint):
* Source/WebCore/page/scrolling/ScrollingCoordinator.cpp:
(WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const):
* Source/WebCore/rendering/HitTestLocation.cpp:
(WebCore::HitTestLocation::HitTestLocation): No longer floor the point.
(WebCore::HitTestLocation::move): No longer convert to IntRect.
* Source/WebCore/rendering/HitTestingTransformState.cpp:
(WebCore::HitTestingTransformState::boundsOfMappedQuad const):
* Source/WebCore/rendering/HitTestingTransformState.h:
* Source/WebCore/rendering/RenderLayer.cpp:
(WebCore::RenderLayer::hitTestLayerByApplyingTransform):
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateAfterWidgetResize):
* Source/WebCore/rendering/RenderLayerCompositor.cpp:
(WebCore::RenderLayerCompositor::frameViewDidChangeLocation):
* Source/WebCore/rendering/RenderLayerCompositor.h:
* Source/WebCore/rendering/RenderWidget.cpp:
(WebCore::RenderWidget::paintContents):
Copy link
Member

@gsnedders gsnedders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments below; also cc @masayuki-nakano, @mustaqahmed, @smaug----.

We probably need to make sure this doesn't affect any of the Pointer Events or UI Events tests, at least.

const rectInView = rectConstrainedToViewport(clientRect);

// Ensure that at least 1px is visible to avoid issues due to elementsFromPoint()
// supporting fractional values, but mouse events flooring to integers.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, them flooring to integers depends on which spec you're looking at, given https://w3c.github.io/uievents/#mouseevent and https://drafts.csswg.org/cssom-view/#extensions-to-the-mouseevent-interface give differing definitions for the same attributes. (See w3c/uievents#24 & w3c/csswg-drafts#4084.)

https://w3c.github.io/pointerevents/#event-coordinates also says "user agents that have implemented the proposed change in CSSOM View Module only for PointerEvent", which implies this doesn't apply to all user agents.

I'm mostly slightly weary about making it impossible to test the Pointer Events behaviour here by introducing rounding always.

Something similar to what we do in this pointer events test might work, checking that at least 1 pixel is in view when fractional values aren't supported, and checking greater than 0 pixels are in view when they are:

if ((new PointerEvent("click", {screenX: 0.1})).screenX == 0.1) {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://w3c.github.io/pointerevents/#event-coordinates also says "user agents that have implemented the proposed change in CSSOM View Module only for PointerEvent", which implies this doesn't apply to all user agents.

If fractional coordinates in click a concern here (I am not sure), we do not need to support that to the best of my knowledge:

  • IIRC, "user agents that have implemented..." was added to cover the unclear "requirement" around CSSOM-View extension: fractional coordinates are neither required not optional because of Converge with CSSOM-View w3c/uievents#24!
  • We (Blink) already confirmed that non-integer coordinates are not web compatible for click-like events.

return new DOMRect(rect.x, rect.y, Math.max(right - left, 0), Math.max(bottom - top, 0));
}

function getElementListAtVisibleCenter(element) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was intended to match https://w3c.github.io/webdriver/#dfn-pointer-interactable-paint-tree, and named as such.

I'm not saying we can't deviate from WebDriver here, but it does hint at there maybe being issues in the WebDriver definitions here.

smfr added a commit to smfr/WebKit that referenced this pull request Jun 18, 2025
https://bugs.webkit.org/show_bug.cgi?id=294571
rdar://152114545

Reviewed by NOBODY (OOPS!).

IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically
painted and hit-tested using integer coordinates. However, iframes are often positioned on
subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an
iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI.

However, we can adjust iframes for painting and hit-testing to overcome the integral
Widget limitation. This PR does this in various places.

First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position,
which fixes painting gaps (tested by iframe-subpixel-painting.html).

Second, the compositing code needs to handle subpixel positioning of the iframe's layers,
fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by
iframe-subpixel-compositing.html.

Finally, hit-testing needs to maintain fractional coordinates;
`documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and
HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the
`nodeAtPoint()` code.

Removing the flooring of the point in the HitTestLocation constructor caused
`css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an
element with a 100x scale. The existing code would map the 1x1px hit-test area through the
scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to
maintain the 1x1px hit-test area in screen space. This is a merge of
https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d

* LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html
fails; web-platform-tests/wpt#53241 exists to fix this.
* LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing-expected.html: Added.
* LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test-expected.txt: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-painting-expected.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-painting.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers
have different results.
* Source/WebCore/page/EventHandler.cpp:
(WebCore::documentPointForWindowPoint):
* Source/WebCore/page/scrolling/ScrollingCoordinator.cpp:
(WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const):
* Source/WebCore/rendering/HitTestLocation.cpp:
(WebCore::HitTestLocation::HitTestLocation): No longer floor the point.
(WebCore::HitTestLocation::move): No longer convert to IntRect.
* Source/WebCore/rendering/HitTestingTransformState.cpp:
(WebCore::HitTestingTransformState::boundsOfMappedQuad const):
* Source/WebCore/rendering/HitTestingTransformState.h:
* Source/WebCore/rendering/RenderLayer.cpp:
(WebCore::RenderLayer::hitTestLayerByApplyingTransform):
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateAfterWidgetResize):
* Source/WebCore/rendering/RenderLayerCompositor.cpp:
(WebCore::RenderLayerCompositor::frameViewDidChangeLocation):
* Source/WebCore/rendering/RenderLayerCompositor.h:
* Source/WebCore/rendering/RenderWidget.cpp:
(WebCore::RenderWidget::paintContents):
webkit-commit-queue pushed a commit to smfr/WebKit that referenced this pull request Jun 19, 2025
https://bugs.webkit.org/show_bug.cgi?id=294571
rdar://152114545

Reviewed by Alan Baradlay.

IFrames are Widgets, and Widgets live in an integer-based hierarchy, so have historically
painted and hit-tested using integer coordinates. However, iframes are often positioned on
subpixel boundaries, which leads to pixel gaps. Some of these are very obvious, e.g. an
iframe inside a `position:fixed; bottom: 0` element that should align seamlessly with UI.

However, we can adjust iframes for painting and hit-testing to overcome the integral
Widget limitation. This PR does this in various places.

First, `RenderWidget::paintContents()` adjusts the CTM to account for a subpixel position,
which fixes painting gaps (tested by iframe-subpixel-painting.html).

Second, the compositing code needs to handle subpixel positioning of the iframe's layers,
fixed in `RenderLayerBacking::updateAfterWidgetResize()` and tested by
iframe-subpixel-compositing.html.

Finally, hit-testing needs to maintain fractional coordinates;
`documentPointForWindowPoint()` is fixed to use FloatPoint coordinate mapping, and
HitTestLocation no longer rounds the origin of m_boundingBox which is what's used by the
`nodeAtPoint()` code.

Removing the flooring of the point in the HitTestLocation constructor caused
`css/css-transforms/transform-scale-hittest.html` to fail; this hit-tests a pixel above an
element with a 100x scale. The existing code would map the 1x1px hit-test area through the
scale, resulting in a 100x100px hit-test area; fix to map through the inverse transform to
maintain the 1x1px hit-test area in screen space. This is a merge of
https://chromium.googlesource.com/chromium/src.git/+/c47a6f0b4a592c2383fb553b6169f89ab01d851d

* LayoutTests/TestExpectations: imported/w3c/web-platform-tests/css/selectors/focus-visible-018-2.html
fails; web-platform-tests/wpt#53241 exists to fix this.
* LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing-expected.html: Added.
* LayoutTests/compositing/iframes/hidpi-iframe-subpixel-compositing.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test-expected.txt: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-hit-test.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-painting-expected.html: Added.
* LayoutTests/fast/frames/hidpi-iframe-subpixel-painting.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/css/cssom-view/elementFromPoint-subpixel-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2-expected.txt:
* LayoutTests/transforms/3d/hit-testing/rotated-hit-test-2.html: Tweak the test; browsers
have different results.
* Source/WebCore/page/EventHandler.cpp:
(WebCore::documentPointForWindowPoint):
* Source/WebCore/page/scrolling/ScrollingCoordinator.cpp:
(WebCore::ScrollingCoordinator::absoluteEventTrackingRegionsForFrame const):
* Source/WebCore/rendering/HitTestLocation.cpp:
(WebCore::HitTestLocation::HitTestLocation): No longer floor the point.
(WebCore::HitTestLocation::move): No longer convert to IntRect.
* Source/WebCore/rendering/HitTestingTransformState.cpp:
(WebCore::HitTestingTransformState::boundsOfMappedQuad const):
* Source/WebCore/rendering/HitTestingTransformState.h:
* Source/WebCore/rendering/RenderLayer.cpp:
(WebCore::RenderLayer::hitTestLayerByApplyingTransform):
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateAfterWidgetResize):
* Source/WebCore/rendering/RenderLayerCompositor.cpp:
(WebCore::RenderLayerCompositor::frameViewDidChangeLocation):
* Source/WebCore/rendering/RenderLayerCompositor.h:
* Source/WebCore/rendering/RenderWidget.cpp:
(WebCore::RenderWidget::paintContents):

Canonical link: https://commits.webkit.org/296407@main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_driver.click() is unreliable because of coordinate rounding issues
5 participants