REGRESSION (259904@main): @math operation cannot trigger in Quip app

https://bugs.webkit.org/show_bug.cgi?id=259944
rdar://113229516

Reviewed by Ryosuke Niwa.

Currently, the `@math` feature in Quip is broken in Safari with Live Range Selection enabled; this
is because Quip's JavaScript editor focuses an input element, and expects `selection.rangeCount` to
return `1` instead of `0` (the latter of which causes Quip to believe that the user is no longer
editing the text field, and therefore leads to Quip dismissing the input field immediately after
presenting it).

This happens because the `rangeCount` and ranges we expose on `DOMSelection` are different with live
range selection in Safari 17, as opposed to both Safari 16 and Chrome (testing against both stable
and canary). Whereas we used to expose a single range that was collapsed to the start of the text
field, we now expose no ranges at all.

To avoid this incompatibility, we restore shipping behavior when the selection is inside of shadow
roots, by exposing a selection range that's equivalent to a caret before the shadow host.

See also: <w3c/selection-api#166>.

Test: fast/forms/shadow-tree-exposure-live-range.html

* LayoutTests/fast/forms/shadow-tree-exposure-live-range-expected.txt:
* LayoutTests/fast/forms/shadow-tree-exposure-live-range.html:

Rebaseline this layout test, such that it verifies that the ranges exposed via the Selection API
have the same behavior without live ranges, vs. with live ranges enabled; this also matches the
behavior in latest Chrome Canary.

* LayoutTests/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt:
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt:
* LayoutTests/platform/ios/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt:

Rebaseline a WPT — attempts to change the selection in a UA shadow root in this test now result in
the wrong base/anchor nodes and offsets after this patch, rather than the `IndexSizeError`s we
currently get.

* Source/WebCore/page/DOMSelection.cpp:
(WebCore::selectionShadowAncestor):

Remove an assertion that's no longer relevant, now that we use this in both live/legacy codepaths.

(WebCore::DOMSelection::type const):

Keep both `fast/forms/shadow-tree-exposure-live-range.html` and
`editing/selection/user-select-js-property.html` passing by adjusting our logic for returning the
selection type. Currently, a selection in a shadow root always results in a type of `None`, but this
diverges from Chrome (and Safari 16's) behavior, which returns the actual type of the selected
range, regardless of whether it's in the shadow tree.

(WebCore::DOMSelection::rangeCount const):

This is the main fix — return a `rangeCount` of 1 in the case where there are no associated live
ranges, if the range is still inside a shadow root.

(WebCore::createLiveRangeBeforeShadowHostWithSelection):
(WebCore::DOMSelection::getRangeAt):

Keep `getRangeAt` consistent with `rangeCount` by returning the caret range at the start of the
shadow host in the case where the selected range is in the shadow tree.

(WebCore::DOMSelection::shadowAdjustedNode const):
(WebCore::DOMSelection::shadowAdjustedOffset const):

These methods need to be adjusted as well, to keep the `(base|anchor|focus|extent)(Node|Offset)`
properties consistent with the `rangeCount` and selection ranges in the case where the selection is
inside of a shadow root. Since the behavior with or without live ranges when the selection is in a
shadow tree is now consistent, we no longer require live-range-specific codepaths, so we can
simplify this code a bit by just removing the `liveRangeSelectionEnabled()` checks.

Canonical link: https://commits.webkit.org/266781@main

LayoutTests/fast/forms/shadow-tree-exposure-live-range-expected.txt

@@ -17,31 +17,31 @@ PASS getSelection().type is "None"
 
 Add an input element.
 
-PASS getSelection().anchorNode is null
-PASS getSelection().anchorOffset is 0
-PASS getSelection().focusNode is null
-PASS getSelection().focusOffset is 0
+PASS getSelection().anchorNode is container
+PASS getSelection().anchorOffset is 1
+PASS getSelection().focusNode is container
+PASS getSelection().focusOffset is 1
 PASS getSelection().isCollapsed is true
-PASS getSelection().rangeCount is 0
-PASS getSelection().baseNode is null
-PASS getSelection().baseOffset is 0
-PASS getSelection().extentNode is null
-PASS getSelection().extentOffset is 0
-PASS getSelection().type is "None"
+PASS getSelection().rangeCount is 1
+PASS getSelection().baseNode is container
+PASS getSelection().baseOffset is 1
+PASS getSelection().extentNode is container
+PASS getSelection().extentOffset is 1
+PASS getSelection().type is "Range"
 
 Add a textarea element.
 
-PASS getSelection().anchorNode is null
-PASS getSelection().anchorOffset is 0
-PASS getSelection().focusNode is null
-PASS getSelection().focusOffset is 0
+PASS getSelection().anchorNode is container
+PASS getSelection().anchorOffset is 2
+PASS getSelection().focusNode is container
+PASS getSelection().focusOffset is 2
 PASS getSelection().isCollapsed is true
-PASS getSelection().rangeCount is 0
-PASS getSelection().baseNode is null
-PASS getSelection().baseOffset is 0
-PASS getSelection().extentNode is null
-PASS getSelection().extentOffset is 0
-PASS getSelection().type is "None"
+PASS getSelection().rangeCount is 1
+PASS getSelection().baseNode is container
+PASS getSelection().baseOffset is 2
+PASS getSelection().extentNode is container
+PASS getSelection().extentOffset is 2
+PASS getSelection().type is "Range"
 
 PASS successfullyParsed is true
 

LayoutTests/fast/forms/shadow-tree-exposure-live-range.html

@@ -35,18 +35,18 @@
 input.focus();
 input.select();
 
-shouldBe("getSelection().anchorNode", "null");
-shouldBe("getSelection().anchorOffset", "0");
-shouldBe("getSelection().focusNode", "null");
-shouldBe("getSelection().focusOffset", "0");
+shouldBe("getSelection().anchorNode", "container");
+shouldBe("getSelection().anchorOffset", "1");
+shouldBe("getSelection().focusNode", "container");
+shouldBe("getSelection().focusOffset", "1");
 shouldBe("getSelection().isCollapsed", "true");
-shouldBe("getSelection().rangeCount", "0");
+shouldBe("getSelection().rangeCount", "1");
 
-shouldBe("getSelection().baseNode", "null");
-shouldBe("getSelection().baseOffset", "0");
-shouldBe("getSelection().extentNode", "null");
-shouldBe("getSelection().extentOffset", "0");
-shouldBeEqualToString("getSelection().type", "None");
+shouldBe("getSelection().baseNode", "container");
+shouldBe("getSelection().baseOffset", "1");
+shouldBe("getSelection().extentNode", "container");
+shouldBe("getSelection().extentOffset", "1");
+shouldBeEqualToString("getSelection().type", "Range");
 
 debug("\nAdd a textarea element.\n");
 
@@ -56,18 +56,18 @@
 textarea.focus();
 textarea.select();
 
-shouldBe("getSelection().anchorNode", "null");
-shouldBe("getSelection().anchorOffset", "0");
-shouldBe("getSelection().focusNode", "null");
-shouldBe("getSelection().focusOffset", "0");
+shouldBe("getSelection().anchorNode", "container");
+shouldBe("getSelection().anchorOffset", "2");
+shouldBe("getSelection().focusNode", "container");
+shouldBe("getSelection().focusOffset", "2");
 shouldBe("getSelection().isCollapsed", "true");
-shouldBe("getSelection().rangeCount", "0");
+shouldBe("getSelection().rangeCount", "1");
 
-shouldBe("getSelection().baseNode", "null");
-shouldBe("getSelection().baseOffset", "0");
-shouldBe("getSelection().extentNode", "null");
-shouldBe("getSelection().extentOffset", "0");
-shouldBeEqualToString("getSelection().type", "None");
+shouldBe("getSelection().baseNode", "container");
+shouldBe("getSelection().baseOffset", "2");
+shouldBe("getSelection().extentNode", "container");
+shouldBe("getSelection().extentOffset", "2");
+shouldBeEqualToString("getSelection().type", "Range");
 
 document.body.removeChild(container);
 

LayoutTests/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt

@@ -8,14 +8,14 @@ PASS Collapse selection into text in the open shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the open shadow DOM
 PASS Set focus to <object> in the open shadow DOM
 PASS Set focus to <p tabindex="0"> in the open shadow DOM
-FAIL SelectAll in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the open shadow DOM assert_in_array: Only all children of the open shadow DOM should be selected value "(<body>, 3)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the open shadow DOM assert_in_array: value "(<body>, 3)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Collapse selection into text in the closed shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the closed shadow DOM
 PASS Set focus to <object> in the closed shadow DOM
 PASS Set focus to <p tabindex="0"> in the closed shadow DOM
-FAIL SelectAll in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the closed shadow DOM assert_in_array: Only all children of the closed shadow DOM should be selected value "(<body>, 7)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the closed shadow DOM assert_in_array: value "(<body>, 7)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Focus after Collapse selection into text in the open shadow DOM
 PASS Typing "A" after Collapse selection into text in the open shadow DOM
 PASS Focus after Collapse selection into text in <div contenteditable> in the open shadow DOM

LayoutTests/platform/glib/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt

@@ -8,14 +8,14 @@ PASS Collapse selection into text in the open shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the open shadow DOM
 PASS Set focus to <object> in the open shadow DOM
 PASS Set focus to <p tabindex="0"> in the open shadow DOM
-FAIL SelectAll in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the open shadow DOM assert_in_array: Only all children of the open shadow DOM should be selected value "(<body>, 3)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the open shadow DOM assert_in_array: value "(<body>, 3)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Collapse selection into text in the closed shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the closed shadow DOM
 PASS Set focus to <object> in the closed shadow DOM
 PASS Set focus to <p tabindex="0"> in the closed shadow DOM
-FAIL SelectAll in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the closed shadow DOM assert_in_array: Only all children of the closed shadow DOM should be selected value "(<body>, 7)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the closed shadow DOM assert_in_array: value "(<body>, 7)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Focus after Collapse selection into text in the open shadow DOM
 PASS Typing "A" after Collapse selection into text in the open shadow DOM
 PASS Focus after Collapse selection into text in <div contenteditable> in the open shadow DOM

LayoutTests/platform/ios/imported/w3c/web-platform-tests/editing/other/editable-state-and-focus-in-shadow-dom-in-designMode.tentative-expected.txt

@@ -8,14 +8,14 @@ PASS Collapse selection into text in the open shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the open shadow DOM
 PASS Set focus to <object> in the open shadow DOM
 PASS Set focus to <p tabindex="0"> in the open shadow DOM
-FAIL SelectAll in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the open shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the open shadow DOM assert_in_array: Only all children of the open shadow DOM should be selected value "(<body>, 3)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the open shadow DOM assert_in_array: value "(<body>, 3)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Collapse selection into text in the closed shadow DOM
 PASS Collapse selection into text in <div contenteditable> in the closed shadow DOM
 PASS Set focus to <object> in the closed shadow DOM
 PASS Set focus to <p tabindex="0"> in the closed shadow DOM
-FAIL SelectAll in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
-FAIL SelectAll in the <div contenteditable> in the closed shadow DOM promise_test: Unhandled rejection with value: object "IndexSizeError: The index is not in the allowed range."
+FAIL SelectAll in the closed shadow DOM assert_in_array: Only all children of the closed shadow DOM should be selected value "(<body>, 7)" not in array ["(#document-fragment, 0) - (#document-fragment, 2)", "(#text \"text\", 0) - (#text \"paragraph\", 9)"]
+FAIL SelectAll in the <div contenteditable> in the closed shadow DOM assert_in_array: value "(<body>, 7)" not in array ["(<div>, 0) - (<div>, 1)", "(#text \"editable\", 0) - (#text \"editable\", 8)"]
 PASS Focus after Collapse selection into text in the open shadow DOM
 PASS Typing "A" after Collapse selection into text in the open shadow DOM
 PASS Focus after Collapse selection into text in <div contenteditable> in the open shadow DOM

Source/WebCore/page/DOMSelection.cpp

@@ -46,7 +46,6 @@ namespace WebCore {
 
 static RefPtr<Node> selectionShadowAncestor(LocalFrame& frame)
 {
-    ASSERT(!frame.settings().liveRangeSelectionEnabled());
     auto* node = frame.selection().selection().base().anchorNode();
     if (!node || !node->isInShadowTree())
         return nullptr;
@@ -180,8 +179,14 @@ String DOMSelection::type() const
     if (!frame)
         return "None"_s;
     auto& selection = frame->selection();
-    if (frame->settings().liveRangeSelectionEnabled())
-        return !selection.isInDocumentTree() ? "None"_s : range()->collapsed() ? "Caret"_s : "Range"_s;
+    if (frame->settings().liveRangeSelectionEnabled()) {
+        auto range = frame->selection().selection().range();
+        if (!range)
+            return "None"_s;
+        if (range->collapsed())
+            return "Caret"_s;
+        return "Range"_s;
+    }
     if (selection.isNone())
         return "None"_s;
     if (selection.isCaret())
@@ -205,8 +210,13 @@ unsigned DOMSelection::rangeCount() const
     RefPtr frame = this->frame();
     if (!frame)
         return 0;
-    if (frame->settings().liveRangeSelectionEnabled())
-        return frame->selection().associatedLiveRange() ? 1 : 0;
+    if (frame->settings().liveRangeSelectionEnabled()) {
+        if (frame->selection().associatedLiveRange())
+            return 1;
+        if (selectionShadowAncestor(*frame))
+            return 1;
+        return 0;
+    }
     return frame->selection().isNone() ? 0 : 1;
 }
 
@@ -386,15 +396,26 @@ ExceptionOr<void> DOMSelection::extend(Node& node, unsigned offset)
     return { };
 }
 
+static RefPtr<Range> createLiveRangeBeforeShadowHostWithSelection(LocalFrame& frame)
+{
+    if (auto shadowAncestor = selectionShadowAncestor(frame))
+        return createLiveRange(makeSimpleRange(*makeBoundaryPointBeforeNode(*shadowAncestor)));
+
+    return nullptr;
+}
+
 ExceptionOr<Ref<Range>> DOMSelection::getRangeAt(unsigned index)
 {
     if (index >= rangeCount())
         return Exception { IndexSizeError };
     auto frame = this->frame().releaseNonNull();
-    if (frame->settings().liveRangeSelectionEnabled())
-        return frame->selection().associatedLiveRange().releaseNonNull();
-    if (auto shadowAncestor = selectionShadowAncestor(frame))
-        return createLiveRange(makeSimpleRange(*makeBoundaryPointBeforeNode(*shadowAncestor)));
+    if (frame->settings().liveRangeSelectionEnabled()) {
+        if (auto liveRange = frame->selection().associatedLiveRange())
+            return liveRange.releaseNonNull();
+        return createLiveRangeBeforeShadowHostWithSelection(frame.get()).releaseNonNull();
+    }
+    if (auto liveRange = createLiveRangeBeforeShadowHostWithSelection(frame.get()))
+        return liveRange.releaseNonNull();
     return createLiveRange(*frame->selection().selection().firstRange());
 }
 
@@ -529,11 +550,6 @@ RefPtr<Node> DOMSelection::shadowAdjustedNode(const Position& position) const
     if (position.isNull())
         return nullptr;
 
-    if (frame()->settings().liveRangeSelectionEnabled()) {
-        auto node = position.containerNode();
-        return !node || node->isInShadowTree() ? nullptr : node;
-    }
-
     auto* containerNode = position.containerNode();
     auto* adjustedNode = frame()->document()->ancestorNodeInThisScope(containerNode);
     if (!adjustedNode)
@@ -550,9 +566,6 @@ unsigned DOMSelection::shadowAdjustedOffset(const Position& position) const
     if (position.isNull())
         return 0;
 
-    if (frame()->settings().liveRangeSelectionEnabled())
-        return shadowAdjustedNode(position) ? position.computeOffsetInContainerNode() : 0;
-
     auto* containerNode = position.containerNode();
     auto* adjustedNode = frame()->document()->ancestorNodeInThisScope(containerNode);
     if (!adjustedNode)