Skip to content
Permalink
Browse files
[Live Text] Use iBeam cursor when hovering over selectable text insid…
…e image links

https://bugs.webkit.org/show_bug.cgi?id=228700
rdar://81210248

Reviewed by Tim Horton.

Source/WebCore:

Make a couple of small adjustments to allow the cursor to change to an I-beam when hovering over selectable Live
Text inside images in links, but only in the case where the computed cursor type is Auto. Currently, this always
results in a Hand cursor type because of the link ancestor, despite the injected Live Text being selectable.

Test: fast/images/text-recognition/mac/cursor-types-for-recognized-text.html

* html/HTMLElement.cpp:
(WebCore::HTMLElement::updateWithTextRecognitionResult):

Drive-by fix an adjacent bug, wherein we try to check whether the image element has `user-select: none;` before
the style has been resolved, which results in injecting selectable Live Text even when the page has explicitly
disabled text selection on the image element. Instead, move this code to right after we update layout after
ensuring the UA shadow DOM structure for Live Text, so that the bool flag is meaningful.

* page/EventHandler.cpp:
(WebCore::EventHandler::selectCursor):

LayoutTests:

Add a layout test that hovers over Live Text in several different image elements, and checks their respective
cursor types.

* fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt: Added.
* fast/images/text-recognition/mac/cursor-types-for-recognized-text.html: Added.


Canonical link: https://commits.webkit.org/240175@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@280548 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
whsieh committed Aug 2, 2021
1 parent ad5c5c0 commit 73ad7e50c27258bc71a8a14811bc9dc2ff96b7f9
Showing 6 changed files with 135 additions and 3 deletions.
@@ -1,3 +1,17 @@
2021-08-02 Wenson Hsieh <wenson_hsieh@apple.com>

[Live Text] Use iBeam cursor when hovering over selectable text inside image links
https://bugs.webkit.org/show_bug.cgi?id=228700
rdar://81210248

Reviewed by Tim Horton.

Add a layout test that hovers over Live Text in several different image elements, and checks their respective
cursor types.

* fast/images/text-recognition/mac/cursor-types-for-recognized-text-expected.txt: Added.
* fast/images/text-recognition/mac/cursor-types-for-recognized-text.html: Added.

2021-08-02 Chris Dumez <cdumez@apple.com>

imported/w3c/web-platform-tests/webmessaging/broadcastchannel/blobs.html is a flaky failure since implementing BlobChannel
@@ -0,0 +1,15 @@

Testing <img id="inside-link">
PASS cursor is "IBeam"

Testing <img id="cursor-zoom-in">
PASS cursor is "ZoomIn"

Testing <img id="user-select-none-in-link">
PASS cursor is "Hand"
PASS successfullyParsed is true

TEST COMPLETE



@@ -0,0 +1,73 @@
<!DOCTYPE html>
<html>
<meta name="viewport" content="width=device-width, initial-scale=1">
<head>
<script src="../../../../resources/js-test.js"></script>
<style>
img {
width: 100px;
height: 100px;
box-sizing: border-box;
border: 1px solid black;
display: block;
}

a {
display: block;
}

#cursor-zoom-in {
top: 100px;
cursor: zoom-in;
}

#user-select-none-in-link {
top: 200px;
-webkit-user-select: none;
}
</style>
<script>
addEventListener("load", () => {
document.querySelectorAll("img").forEach(image => {
internals.installImageOverlay(image, [
{
topLeft : new DOMPointReadOnly(0, 0.3),
topRight : new DOMPointReadOnly(1, 0.3),
bottomRight : new DOMPointReadOnly(1, 0.7),
bottomLeft : new DOMPointReadOnly(0, 0.7),
children: [{
text : "Hello",
topLeft : new DOMPointReadOnly(0, 0.3),
topRight : new DOMPointReadOnly(1, 0.3),
bottomRight : new DOMPointReadOnly(1, 0.7),
bottomLeft : new DOMPointReadOnly(0, 0.7),
}],
}
]);
});

function testImage(imageID, expectedResult) {
let image = document.getElementById(imageID);
let rect = image.getBoundingClientRect();
eventSender.mouseMoveTo(rect.left + rect.width / 2, rect.top + rect.height / 2);
cursor = internals.getCurrentCursorInfo().match(/type=(\w+)\s/).pop();
debug(`\nTesting &lt;img id="${imageID}"&gt;`);
shouldBeEqualToString("cursor", expectedResult);
}

testImage("inside-link", "IBeam");
testImage("cursor-zoom-in", "ZoomIn");
testImage("user-select-none-in-link", "Hand");
});
</script>
</head>
<body>
<a href="#">
<img id="inside-link" src="../../resources/green-400x400.png"></img>
</a>
<img id="cursor-zoom-in" src="../../resources/green-400x400.png"></img>
<a href="#">
<img id="user-select-none-in-link" src="../../resources/green-400x400.png"></img>
</a>
</body>
</html>
@@ -1,3 +1,28 @@
2021-08-02 Wenson Hsieh <wenson_hsieh@apple.com>

[Live Text] Use iBeam cursor when hovering over selectable text inside image links
https://bugs.webkit.org/show_bug.cgi?id=228700
rdar://81210248

Reviewed by Tim Horton.

Make a couple of small adjustments to allow the cursor to change to an I-beam when hovering over selectable Live
Text inside images in links, but only in the case where the computed cursor type is Auto. Currently, this always
results in a Hand cursor type because of the link ancestor, despite the injected Live Text being selectable.

Test: fast/images/text-recognition/mac/cursor-types-for-recognized-text.html

* html/HTMLElement.cpp:
(WebCore::HTMLElement::updateWithTextRecognitionResult):

Drive-by fix an adjacent bug, wherein we try to check whether the image element has `user-select: none;` before
the style has been resolved, which results in injecting selectable Live Text even when the page has explicitly
disabled text selection on the image element. Instead, move this code to right after we update layout after
ensuring the UA shadow DOM structure for Live Text, so that the bool flag is meaningful.

* page/EventHandler.cpp:
(WebCore::EventHandler::selectCursor):

2021-08-02 Chris Dumez <cdumez@apple.com>

imported/w3c/web-platform-tests/webmessaging/broadcastchannel/blobs.html is a flaky failure since implementing BlobChannel
@@ -1424,7 +1424,6 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
return;

auto shadowRoot = makeRef(ensureUserAgentShadowRoot());
bool hasUserSelectNone = renderer() && renderer()->style().userSelect() == UserSelect::None;
if (!textRecognitionElements.root) {
auto rootContainer = HTMLDivElement::create(document());
rootContainer->setIdAttribute(imageOverlayElementIdentifier());
@@ -1489,6 +1488,7 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
return quad;
};

bool applyUserSelectAll = document().isImageDocument() || renderer->style().userSelect() != UserSelect::None;
for (size_t lineIndex = 0; lineIndex < result.lines.size(); ++lineIndex) {
auto& lineElements = textRecognitionElements.lines[lineIndex];
auto& lineContainer = lineElements.line;
@@ -1572,8 +1572,7 @@ void HTMLElement::updateWithTextRecognitionResult(const TextRecognitionResult& r
"scale("_s, targetSize.width() / sizeBeforeTransform.width(), ", "_s, targetSize.height() / sizeBeforeTransform.height(), ") "_s
));

if (!hasUserSelectNone || document().isImageDocument())
textContainer->setInlineStyleProperty(CSSPropertyWebkitUserSelect, CSSValueAll);
textContainer->setInlineStyleProperty(CSSPropertyWebkitUserSelect, applyUserSelectAll ? CSSValueAll : CSSValueNone);
}

if (document().isImageDocument())
@@ -1509,6 +1509,12 @@ std::optional<Cursor> EventHandler::selectCursor(const HitTestResult& result, bo

switch (style ? style->cursor() : CursorType::Auto) {
case CursorType::Auto: {
if (HTMLElement::isImageOverlayText(node.get())) {
auto* renderer = node->renderer();
if (renderer && renderer->style().userSelect() != UserSelect::None)
return iBeam;
}

bool editable = node->hasEditableStyle();

if (useHandCursor(node.get(), result.isOverLink(), shiftKey))

0 comments on commit 73ad7e5

Please sign in to comment.