Skip to content

Commit

Permalink
Throttle mousemove events to one per rendering update
Browse files Browse the repository at this point in the history
https://bugs.webkit.org/show_bug.cgi?id=259408
rdar://110921187

Reviewed by Tim Horton and Simon Fraser.

Throttle the mousemove event dispatch rate to a maximum of 1 per rendering update, if the user isn't
clicking or dragging. See below for more details.

Test:   MouseEventTests.CoalesceMouseMoveEvents
        MouseEventTests.ProcessSwapWithDeferredMouseMoveEventCompletion

* LayoutTests/fast/selectors/style-invalidation-hover-change-descendants-expected.txt:
* LayoutTests/fast/selectors/style-invalidation-hover-change-descendants.html:
* LayoutTests/fast/selectors/style-invalidation-hover-change-siblings-expected.txt:
* LayoutTests/fast/selectors/style-invalidation-hover-change-siblings.html:

Adjust a couple of layout tests that need to run checks for style invalidation state immediately
after handling the mousemove event, without waiting for any further IPC messages to be dispatched.
Ensure this by moving the test logic into one-shot `mousemove` event listeners; we also take the
opportunity to modernize the test a bit by using `js-test.js` and `testPassed` / `testFailed`.

* Source/WebKit/Shared/WebEvent.h:
* Source/WebKit/Shared/WebEventType.h: Copied from Source/WebKit/Shared/WebEvent.h.
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::handleMouseEvent):

See comments under `WebPage::mouseEvent`.

* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/InjectedBundle/API/c/WKBundlePage.cpp:
(WKBundlePageFlushDeferredDidReceiveMouseEventForTesting):

Add a testing-only SPI hook to flush the `DidReceiveEvent` message corresponding to any `mousemove`
events that have already been handled in the web process. WebKitTestRunner uses this to ensure that
`window.eventSender` API to simulate mouse events behaves the same way as it currently does.

* Source/WebKit/WebProcess/InjectedBundle/API/c/WKBundlePage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::close):
(WebKit::WebPage::suspendForProcessSwap):

Flush any deferred `mousemove` IPC responses here when suspending right before a process swap, so
that we won't end up trying to send a `DidReceiveEvent` message back to the UI process when the
`WebPageProxy`'s `mouseEventQueue` has already been emptied, due to process swapping.

(WebKit::WebPage::mouseEvent):

When handling a `mousemove` event, rather than invoke the IPC completion handler immediately, we
instead defer the call to `WebPageProxy::DidReceiveEvent` until the end of the current rendering
update. This means that existing UI-side logic for coalescing `mousemove` events when the web
process is still handling mouse events will coalesce mousemove events until the end of the rendering
update, after which the web process will be done with current mousemove.

If a non-`mousemove` event enters the mouse event queue), we'll also tell the web process to
dispatch the deferred mousemove completion handler early so that we can process the incoming click
right away.

(WebKit::WebPage::flushDeferredDidReceiveMouseEvent):
(WebKit::WebPage::finalizeRenderingUpdate):

Clear out `m_deferredDidReceiveMouseEvent` if it was set, and dispatch the `DidReceiveEvent` message
back to the UI process for this deferred event.

(WebKit::WebPage::didCommitLoad):

Similarly, flush any deferred `DidReceiveEvent` response when committing a load.

* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.messages.in:
* Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* Tools/TestWebKitAPI/Tests/mac/MouseEventTests.mm: Added.

Add an API test to exercise `mousemove` event coalescing (both this new completion deferral, as well
as the preexisting mechanism for coalescing events in the UI process). This test simulates moving a
mouse cursor over (300, 300) in the window, clicking, and then verifies that:

1.  There are at least 4 mouse events observed: a `mousemove` for the starting location, another
    `mousemove` for the final destination, and `mousedown` and `mouseup` events for the click.

2.  There are no more than 200 events in total (in other words, some `mousemove` events were
    coalesced).

Also, add an API test to verify that we don't hit a `MESSAGE_CHECK` due to sending a deferred
`mousemove` event completion message during a process swap.

* Tools/WebKitTestRunner/InjectedBundle/EventSendingController.cpp:
(WTR::EventSendingController::mouseMoveTo):

Use the above testing hook. We also wait for one extra web <-> UI process round trip here in order
to ensure that the UI process has received the message from the web process indicating that we've
handled the `mousemove` event.

* Tools/WebKitTestRunner/TestController.cpp:
(WTR::TestController::didReceiveSynchronousMessageFromInjectedBundle):

Canonical link: https://commits.webkit.org/266341@main
  • Loading branch information
whsieh committed Jul 27, 2023
1 parent 0d85381 commit a10c481
Show file tree
Hide file tree
Showing 17 changed files with 359 additions and 81 deletions.
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
PASS
PASS
PASS
PASS
PASS
PASS
PASS .target styleChangeType was InlineStyleChange
PASS .container styleChangeType was NoStyleChange
PASS .inert styleChangeType was NoStyleChange
PASS .target styleChangeType was InlineStyleChange
PASS .container styleChangeType was NoStyleChange
PASS .inert styleChangeType was NoStyleChange
PASS successfullyParsed is true

TEST COMPLETE

Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
<html>
<head>
<script src="../../resources/js-test.js"></script>
<script src="../../resources/ui-helper.js"></script>
</head>
<style>
.container {
background-color: blue;
Expand Down Expand Up @@ -42,7 +46,6 @@
</div>
</div>
</div>
<pre id=log></pre>

<script>
function testStyleChangeType(selector, expectedType)
Expand All @@ -52,31 +55,35 @@
for (var i = 0; i < elements.length; ++i) {
const type = window.internals.styleChangeType(elements[i]);
if (type != expectedType) {
log.textContent += `FAIL ${selector} styleChangeType was ${type} expected ${expectedType}\n`;
testFailed(`${selector} styleChangeType was ${type} expected ${expectedType}`);
pass = false;
}
}
if (pass)
log.textContent += "PASS\n";
testPassed(`${selector} styleChangeType was ${expectedType}`);
}

window.onload = function () {
jsTestIsAsync = true;

window.onload = async () => {
if (!window.testRunner)
return;
testRunner.dumpAsText();

document.body.offsetLeft;
eventSender.mouseMoveTo(50,50);

testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
addEventListener("mousemove", () => {
testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
}, { "once" : true });
eventSender.mouseMoveTo(50, 50);

document.body.offsetLeft;
eventSender.mouseMoveTo(300,50);
await UIHelper.ensurePresentationUpdate();
addEventListener("mousemove", () => {
testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
}, { "once" : true });
eventSender.mouseMoveTo(300, 50, "mouse", false);

testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
finishJSTest();
};
</script>
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
PASS
PASS
PASS
PASS
PASS
PASS
PASS .target styleChangeType was InlineStyleChange
PASS .container styleChangeType was NoStyleChange
PASS .inert styleChangeType was NoStyleChange
PASS .target styleChangeType was InlineStyleChange
PASS .container styleChangeType was NoStyleChange
PASS .inert styleChangeType was NoStyleChange
PASS successfullyParsed is true

TEST COMPLETE

Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
<html>
<head>
<script src="../../resources/js-test.js"></script>
<script src="../../resources/ui-helper.js"></script>
</head>
<style>
.container {
background-color: blue;
Expand Down Expand Up @@ -45,7 +49,6 @@
</div>
</div>
</div>
<pre id=log></pre>

<script>
function testStyleChangeType(selector, expectedType)
Expand All @@ -55,31 +58,36 @@
for (var i = 0; i < elements.length; ++i) {
const type = window.internals.styleChangeType(elements[i]);
if (type != expectedType) {
log.textContent += `FAIL ${selector} styleChangeType was ${type} expected ${expectedType}\n`;
testFailed(`${selector} styleChangeType was ${type} expected ${expectedType}`);
pass = false;
}
}
if (pass)
log.textContent += "PASS\n";
testPassed(`${selector} styleChangeType was ${expectedType}`);
}

window.onload = function () {
jsTestIsAsync = true;

window.onload = async () => {
if (!window.testRunner)
return;
testRunner.dumpAsText();

document.body.offsetLeft;
eventSender.mouseMoveTo(50,50);
addEventListener("mousemove", () => {
testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
}, { "once" : true });
eventSender.mouseMoveTo(50, 50);

testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
await UIHelper.ensurePresentationUpdate();

document.body.offsetLeft;
eventSender.mouseMoveTo(300,50);
addEventListener("mousemove", () => {
testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
}, { "once" : true });
eventSender.mouseMoveTo(300, 50);

testStyleChangeType(".target", "InlineStyleChange");
testStyleChangeType(".container", "NoStyleChange");
testStyleChangeType(".inert", "NoStyleChange");
finishJSTest();
};
</script>
36 changes: 1 addition & 35 deletions Source/WebKit/Shared/WebEvent.h
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@

#include "WebEvent.h"
#include "WebEventModifier.h"
#include "WebEventType.h"

#include <wtf/EnumTraits.h>
#include <wtf/OptionSet.h>
Expand All @@ -46,41 +47,6 @@ class Encoder;

namespace WebKit {

enum class WebEventType : int8_t {
NoType = -1,

// WebMouseEvent
MouseDown,
MouseUp,
MouseMove,
MouseForceChanged,
MouseForceDown,
MouseForceUp,

// WebWheelEvent
Wheel,

// WebKeyboardEvent
KeyDown,
KeyUp,
RawKeyDown,
Char,

#if ENABLE(TOUCH_EVENTS)
// WebTouchEvent
TouchStart,
TouchMove,
TouchEnd,
TouchCancel,
#endif

#if ENABLE(MAC_GESTURE_EVENTS)
GestureStart,
GestureChange,
GestureEnd,
#endif
};

class WebEvent {
WTF_MAKE_FAST_ALLOCATED;
public:
Expand Down
65 changes: 65 additions & 0 deletions Source/WebKit/Shared/WebEventType.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright (C) 2023 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*/

#pragma once

namespace WebKit {

enum class WebEventType : int8_t {
NoType = -1,

// WebMouseEvent
MouseDown,
MouseUp,
MouseMove,
MouseForceChanged,
MouseForceDown,
MouseForceUp,

// WebWheelEvent
Wheel,

// WebKeyboardEvent
KeyDown,
KeyUp,
RawKeyDown,
Char,

#if ENABLE(TOUCH_EVENTS)
// WebTouchEvent
TouchStart,
TouchMove,
TouchEnd,
TouchCancel,
#endif

#if ENABLE(MAC_GESTURE_EVENTS)
GestureStart,
GestureChange,
GestureEnd,
#endif
};

} // namespace WebKit
3 changes: 3 additions & 0 deletions Source/WebKit/UIProcess/WebPageProxy.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3210,6 +3210,9 @@ void WebPageProxy::handleMouseEvent(const NativeWebMouseEvent& event)
UNUSED_PARAM(didRemoveEvent);
LOG_WITH_STREAM(MouseHandling, stream << "UIProcess: " << (didRemoveEvent ? "replaced" : "enqueued") << " mouse event " << event.type() << " (queue size " << internals().mouseEventQueue.size() << ")");

if (event.type() != WebEventType::MouseMove)
send(Messages::WebPage::FlushDeferredDidReceiveMouseEvent());

if (internals().mouseEventQueue.size() == 1) // Otherwise, called from DidReceiveEvent message handler.
processNextQueuedMouseEvent();
}
Expand Down
4 changes: 4 additions & 0 deletions Source/WebKit/WebKit.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -2205,6 +2205,7 @@
F4BA33F225757E89000A3CE8 /* WKImageAnalysisGestureRecognizer.h in Headers */ = {isa = PBXBuildFile; fileRef = F4BA33F025757E89000A3CE8 /* WKImageAnalysisGestureRecognizer.h */; };
F4BE0D7727AAE1CB005F0323 /* PlaybackSessionContextIdentifier.h in Headers */ = {isa = PBXBuildFile; fileRef = F4BE0D7627AAE1CB005F0323 /* PlaybackSessionContextIdentifier.h */; };
F4CB09E5225D5A0900891487 /* WebsiteMediaSourcePolicy.h in Headers */ = {isa = PBXBuildFile; fileRef = F4CB09E4225D5A0300891487 /* WebsiteMediaSourcePolicy.h */; };
F4CBF79E2A6ADF7E00C066BF /* WebEventType.h in Headers */ = {isa = PBXBuildFile; fileRef = F4CBF79D2A6ADF5400C066BF /* WebEventType.h */; };
F4CF1E9D25E40DCC000F9D73 /* GestureRecognizerConsistencyEnforcer.h in Headers */ = {isa = PBXBuildFile; fileRef = F4CF1E9B25E40DCC000F9D73 /* GestureRecognizerConsistencyEnforcer.h */; };
F4D188EF29B99B4500D838D4 /* PrivateClickMeasurementEphemeralStore.h in Headers */ = {isa = PBXBuildFile; fileRef = F4D188ED29B99B4500D838D4 /* PrivateClickMeasurementEphemeralStore.h */; };
F4D5F51D206087A10038BBA8 /* WKTextInputListViewController.h in Headers */ = {isa = PBXBuildFile; fileRef = F4D5F519206087A00038BBA8 /* WKTextInputListViewController.h */; };
Expand Down Expand Up @@ -7237,6 +7238,7 @@
F4BA33F125757E89000A3CE8 /* WKImageAnalysisGestureRecognizer.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; name = WKImageAnalysisGestureRecognizer.mm; path = ios/WKImageAnalysisGestureRecognizer.mm; sourceTree = "<group>"; };
F4BE0D7627AAE1CB005F0323 /* PlaybackSessionContextIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PlaybackSessionContextIdentifier.h; sourceTree = "<group>"; };
F4CB09E4225D5A0300891487 /* WebsiteMediaSourcePolicy.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WebsiteMediaSourcePolicy.h; sourceTree = "<group>"; };
F4CBF79D2A6ADF5400C066BF /* WebEventType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WebEventType.h; sourceTree = "<group>"; };
F4CF1E9B25E40DCC000F9D73 /* GestureRecognizerConsistencyEnforcer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = GestureRecognizerConsistencyEnforcer.h; path = ios/GestureRecognizerConsistencyEnforcer.h; sourceTree = "<group>"; };
F4CF1E9C25E40DCC000F9D73 /* GestureRecognizerConsistencyEnforcer.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; name = GestureRecognizerConsistencyEnforcer.mm; path = ios/GestureRecognizerConsistencyEnforcer.mm; sourceTree = "<group>"; };
F4D188ED29B99B4500D838D4 /* PrivateClickMeasurementEphemeralStore.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PrivateClickMeasurementEphemeralStore.h; sourceTree = "<group>"; };
Expand Down Expand Up @@ -8113,6 +8115,7 @@
BC032DB110F4380F0058C15A /* WebEventConversion.h */,
33D059A42A1EEEDC009AFE71 /* WebEventModifier.cpp */,
86DD518F28EF28E800DF2A58 /* WebEventModifier.h */,
F4CBF79D2A6ADF5400C066BF /* WebEventType.h */,
1C0234BF28A00DCF00AC1E5B /* WebExtensionContextIdentifier.h */,
1C0234BE28A00DCF00AC1E5B /* WebExtensionContextParameters.h */,
1CBEE26128F334D6006D1A02 /* WebExtensionContextParameters.serialization.in */,
Expand Down Expand Up @@ -14875,6 +14878,7 @@
BC032DBB10F4380F0058C15A /* WebEventConversion.h in Headers */,
BC111B5D112F629800337BAB /* WebEventFactory.h in Headers */,
86DD519028EF28E800DF2A58 /* WebEventModifier.h in Headers */,
F4CBF79E2A6ADF7E00C066BF /* WebEventType.h in Headers */,
B6544F932937E45100034EB0 /* WebExtensionAPIEvent.h in Headers */,
1C5DC46D290B271E0061EC62 /* WebExtensionAPIExtension.h in Headers */,
1C5DC46C290B271E0061EC62 /* WebExtensionAPINamespace.h in Headers */,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -813,6 +813,11 @@ void WKBundlePageCallAfterTasksAndTimers(WKBundlePageRef pageRef, WKBundlePageTe
});
}

void WKBundlePageFlushDeferredDidReceiveMouseEventForTesting(WKBundlePageRef page)
{
WebKit::toImpl(page)->flushDeferredDidReceiveMouseEvent();
}

void WKBundlePagePostMessage(WKBundlePageRef pageRef, WKStringRef messageNameRef, WKTypeRef messageBodyRef)
{
WebKit::toImpl(pageRef)->postMessage(WebKit::toWTFString(messageNameRef), WebKit::toImpl(messageBodyRef));
Expand Down
2 changes: 2 additions & 0 deletions Source/WebKit/WebProcess/InjectedBundle/API/c/WKBundlePage.h
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ WK_EXPORT void WKBundlePagePostSynchronousMessageForTesting(WKBundlePageRef page
// Same as WKBundlePagePostMessage() but the message cannot become synchronous, even if the connection is in fully synchronous mode.
WK_EXPORT void WKBundlePagePostMessageIgnoringFullySynchronousMode(WKBundlePageRef page, WKStringRef messageName, WKTypeRef messageBody);

WK_EXPORT void WKBundlePageFlushDeferredDidReceiveMouseEventForTesting(WKBundlePageRef page);

#ifdef __cplusplus
}
#endif
Expand Down
Loading

0 comments on commit a10c481

Please sign in to comment.