[Live Text] Use iBeam cursor when hovering over selectable text insid…

…e image links https://bugs.webkit.org/show_bug.cgi?id=228700 rdar://81210248 Reviewed by Tim Horton. Source/WebCore: Make a couple of small adjustments to allow the cursor to change to an I-beam when hovering over selectable Live Text inside images in links, but only in the case where the computed cursor type is Auto. Currently, this always results in a Hand cursor type because of the link ancestor, despite the injected Live Text being selectable. Test: fast/images/text-recognition/mac/cursor-types-for-recognized-text.html * html/HTMLElement.cpp: (WebCore::HTMLElement::updateWithTextRecognitionResult): Drive-by fix an adjacent bug, wherein we try to check whether the image element has `user-select: none;` before the style has been resolved, which results in injecting selectable Live Text even when the page has explicitly disabled text selection on the image element. Instead, move this code to right after we update layout after ensuring the UA shadow DOM structure for Live Text, so that the bool flag is meaningful. * page/EventHandler.cpp: (WebCore::EventHandler::selectCursor): LayoutTests: Add a layout test that hovers over Live Text in several different image elements, and checks their respective cursor types. * fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt: Added. * fast/images/text-recognition/mac/cursor-types-for-recognized-text.html: Added. Canonical link: https://commits.webkit.org/240175@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@280548 268f45cc-cd09-0410-ab3c-d52691b4dbfc
WebKit · Aug 2, 2021 · 73ad7e50c27258bc71a8a14811bc9dc2ff96b7f9 · 73ad7e5
1 parent ad5c5c0
commit 73ad7e50c27258bc71a8a14811bc9dc2ff96b7f9
Showing 6 changed files with 135 additions and 3 deletions.
diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog
@@ -1,3 +1,17 @@
+2021-08-02  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Live Text] Use iBeam cursor when hovering over selectable text inside image links
+        https://bugs.webkit.org/show_bug.cgi?id=228700
+        rdar://81210248
+
+        Reviewed by Tim Horton.
+
+        Add a layout test that hovers over Live Text in several different image elements, and checks their respective
+        cursor types.
+
+        * fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt: Added.
+        * fast/images/text-recognition/mac/cursor-types-for-recognized-text.html: Added.
+
 2021-08-02  Chris Dumez  <cdumez@apple.com>
 
         imported/w3c/web-platform-tests/webmessaging/broadcastchannel/blobs.html is a flaky failure since implementing BlobChannel

diff --git a/LayoutTests/fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt b/LayoutTests/fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt
@@ -0,0 +1,15 @@
+
+Testing <img id="inside-link">
+PASS cursor is "IBeam"
+
+Testing <img id="cursor-zoom-in">
+PASS cursor is "ZoomIn"
+
+Testing <img id="user-select-none-in-link">
+PASS cursor is "Hand"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+
+
diff --git a/LayoutTests/fast/images/text-recognition/mac/cursor-types-for-recognized-text.html b/LayoutTests/fast/images/text-recognition/mac/cursor-types-for-recognized-text.html
@@ -0,0 +1,73 @@
+<!DOCTYPE html>
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<head>
+<script src="../../../../resources/js-test.js"></script>
+<style>
+img {
+    width: 100px;
+    height: 100px;
+    box-sizing: border-box;
+    border: 1px solid black;
+    display: block;
+}
+
+a {
+    display: block;
+}
+
+#cursor-zoom-in {
+    top: 100px;
+    cursor: zoom-in;
+}
+
+#user-select-none-in-link {
+    top: 200px;
+    -webkit-user-select: none;
+}
+</style>
+<script>
+addEventListener("load", () => {
+    document.querySelectorAll("img").forEach(image => {
+        internals.installImageOverlay(image, [
+            {
+                topLeft : new DOMPointReadOnly(0, 0.3),
+                topRight : new DOMPointReadOnly(1, 0.3),
+                bottomRight : new DOMPointReadOnly(1, 0.7),
+                bottomLeft : new DOMPointReadOnly(0, 0.7),
+                children: [{
+                    text : "Hello",
+                    topLeft : new DOMPointReadOnly(0, 0.3),
+                    topRight : new DOMPointReadOnly(1, 0.3),
+                    bottomRight : new DOMPointReadOnly(1, 0.7),
+                    bottomLeft : new DOMPointReadOnly(0, 0.7),
+                }],
+            }
+        ]);
+    });
+
+    function testImage(imageID, expectedResult) {
+        let image = document.getElementById(imageID);
+        let rect = image.getBoundingClientRect();
+        eventSender.mouseMoveTo(rect.left + rect.width / 2, rect.top + rect.height / 2);
+        cursor = internals.getCurrentCursorInfo().match(/type=(\w+)\s/).pop();
+        debug(`\nTesting &lt;img id="${imageID}"&gt;`);
+        shouldBeEqualToString("cursor", expectedResult);
+    }
+
+    testImage("inside-link", "IBeam");
+    testImage("cursor-zoom-in", "ZoomIn");
+    testImage("user-select-none-in-link", "Hand");
+});
+</script>
+</head>
+<body>
+<a href="#">
+    <img id="inside-link" src="../../resources/green-400x400.png"></img>
+</a>
+<img id="cursor-zoom-in" src="../../resources/green-400x400.png"></img>
+<a href="#">
+    <img id="user-select-none-in-link" src="../../resources/green-400x400.png"></img>
+</a>
+</body>
+</html>
diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
@@ -1,3 +1,28 @@
+2021-08-02  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [Live Text] Use iBeam cursor when hovering over selectable text inside image links
+        https://bugs.webkit.org/show_bug.cgi?id=228700
+        rdar://81210248
+
+        Reviewed by Tim Horton.
+
+        Make a couple of small adjustments to allow the cursor to change to an I-beam when hovering over selectable Live
+        Text inside images in links, but only in the case where the computed cursor type is Auto. Currently, this always
+        results in a Hand cursor type because of the link ancestor, despite the injected Live Text being selectable.
+
+        Test: fast/images/text-recognition/mac/cursor-types-for-recognized-text.html
+
+        * html/HTMLElement.cpp:
+        (WebCore::HTMLElement::updateWithTextRecognitionResult):
+
+        Drive-by fix an adjacent bug, wherein we try to check whether the image element has `user-select: none;` before
+        the style has been resolved, which results in injecting selectable Live Text even when the page has explicitly
+        disabled text selection on the image element. Instead, move this code to right after we update layout after
+        ensuring the UA shadow DOM structure for Live Text, so that the bool flag is meaningful.
+
+        * page/EventHandler.cpp:
+        (WebCore::EventHandler::selectCursor):
+
 2021-08-02  Chris Dumez  <cdumez@apple.com>
 
         imported/w3c/web-platform-tests/webmessaging/broadcastchannel/blobs.html is a flaky failure since implementing BlobChannel

diff --git a/Source/WebCore/html/HTMLElement.cpp b/Source/WebCore/html/HTMLElement.cpp
@@ -1424,7 +1424,6 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
         return;
 
     auto shadowRoot = makeRef(ensureUserAgentShadowRoot());
-    bool hasUserSelectNone = renderer() && renderer()->style().userSelect() == UserSelect::None;
     if (!textRecognitionElements.root) {
         auto rootContainer = HTMLDivElement::create(document());
         rootContainer->setIdAttribute(imageOverlayElementIdentifier());
@@ -1489,6 +1488,7 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
         return quad;
     };
 
+    bool applyUserSelectAll = document().isImageDocument() || renderer->style().userSelect() != UserSelect::None;
     for (size_t lineIndex = 0; lineIndex < result.lines.size(); ++lineIndex) {
         auto& lineElements = textRecognitionElements.lines[lineIndex];
         auto& lineContainer = lineElements.line;
@@ -1572,8 +1572,7 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
                 "scale("_s, targetSize.width() / sizeBeforeTransform.width(), ", "_s, targetSize.height() / sizeBeforeTransform.height(), ") "_s
             ));
 
-            if (!hasUserSelectNone || document().isImageDocument())
-                textContainer->setInlineStyleProperty(CSSPropertyWebkitUserSelect, CSSValueAll);
+            textContainer->setInlineStyleProperty(CSSPropertyWebkitUserSelect, applyUserSelectAll ? CSSValueAll : CSSValueNone);
         }
 
         if (document().isImageDocument())

diff --git a/Source/WebCore/page/EventHandler.cpp b/Source/WebCore/page/EventHandler.cpp
@@ -1509,6 +1509,12 @@ std::optional<Cursor> EventHandler::selectCursor(const HitTestResult& result, bo
 
     switch (style ? style->cursor() : CursorType::Auto) {
     case CursorType::Auto: {
+        if (HTMLElement::isImageOverlayText(node.get())) {
+            auto* renderer = node->renderer();
+            if (renderer && renderer->style().userSelect() != UserSelect::None)
+                return iBeam;
+        }
+
         bool editable = node->hasEditableStyle();
 
         if (useHandCursor(node.get(), result.isOverLink(), shiftKey))