Skip to content
Permalink
Browse files
AX: WKContentView needs to implement UITextInput methods to make spea…
…k selection highlighting work

https://bugs.webkit.org/show_bug.cgi?id=166955

Reviewed by Ryosuke Niwa.

Source/WebCore:

Created a new version of Range::collectSelectionRect() that returns rects for each
line, so that Speak Selection doesn't need to handle searching for soft line breaks.
Also added a variant of findPlainText to search for the closest matched range to the given position.

Test: editing/text-iterator/range-of-string-closest-to-position.html

* dom/Range.cpp:
(WebCore::Range::collectSelectionRectsWithoutUnionInteriorLines):
(WebCore::Range::collectSelectionRects):
* dom/Range.h:
* editing/TextIterator.cpp:
(WebCore::findPlainTextMatches):
(WebCore::updateSearchBuffer):
(WebCore::findIteratorOptions):
(WebCore::rangeMatches):
(WebCore::findClosestPlainText):
(WebCore::findPlainText):
(WebCore::findPlainTextOffset): Deleted.
* editing/TextIterator.h:
* editing/htmlediting.h:
* testing/Internals.cpp:
(WebCore::Internals::rangeOfStringNearLocation):
* testing/Internals.h:
* testing/Internals.idl:

Source/WebKit2:

Implemented methods that Speak Selection can use to retrieve the word/sentence highlighting rects.

* Scripts/webkit/messages.py:
(headers_for_type):
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView webSelectionRectsForSelectionRects:]):
(-[WKContentView webSelectionRects]):
(-[WKContentView _accessibilityRetrieveRectsEnclosingSelectionOffset:withGranularity:]):
(-[WKContentView _accessibilityRetrieveRectsAtSelectionOffset:withText:]):
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::selectionRectsCallback):
(WebKit::WebPageProxy::requestRectsForGranularityWithSelectionOffset):
(WebKit::WebPageProxy::requestRectsAtSelectionOffsetWithText):
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::visiblePositionForPositionWithOffset):
(WebKit::WebPage::getRectsForGranularityWithSelectionOffset):
(WebKit::rangeNearPositionMatchesText):
(WebKit::WebPage::getRectsAtSelectionOffsetWithText):

LayoutTests:

* editing/text-iterator/range-of-string-closest-to-position-expected.txt: Added.
* editing/text-iterator/range-of-string-closest-to-position.html: Added.


Canonical link: https://commits.webkit.org/184600@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@211356 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
Nan Wang committed Jan 30, 2017
1 parent ee0cf0b commit be98853be8bdbef0bd342c41c5c2c7547b57fa97
@@ -1,3 +1,13 @@
2017-01-29 Nan Wang <n_wang@apple.com>

AX: WKContentView needs to implement UITextInput methods to make speak selection highlighting work
https://bugs.webkit.org/show_bug.cgi?id=166955

Reviewed by Ryosuke Niwa.

* editing/text-iterator/range-of-string-closest-to-position-expected.txt: Added.
* editing/text-iterator/range-of-string-closest-to-position.html: Added.

2017-01-29 Yoav Weiss <yoav@yoav.ws>

Add invalid value tests to Link header handling.
@@ -0,0 +1,11 @@
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), 'the', 11).toArray() is secondThe
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), 'the', 9).toArray() is secondThe
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), 'the', 6).toArray() is firstThe
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), 'the', 16).toArray() is secondThe
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), 'The', 11).toArray() is upperThe
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p2), '中文', 5).toArray() is chineseMatch
PASS internals.rangeOfStringNearLocation(rangeOfNodeContent(p3), 'the', 5).toArray() is rtlMatch
PASS successfullyParsed is true

TEST COMPLETE

@@ -0,0 +1,75 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<script src="../../resources/js-test-pre.js"></script>
</head>
<body>
<div id="test" contenteditable>
<p id="p1">the car is the good the The world.</p>
<p id="p2">中文字是中文字</p>
<bdo dir="rtl" id="p3">the best</bdo>
</div>
<div id="console"></div>
<script>

if (!window.internals)
testFailed('This test requires internals object');
else {
var container = document.getElementById('test');

function range(startContainer, startOffset, endContainer, endOffset) {
var range = document.createRange();
range.setStart(startContainer, startOffset);
range.setEnd(endContainer, endOffset);
return range;
}

function rangeOfNodeContent(referenceNode) {
var range = document.createRange();
range.selectNodeContents(referenceNode);
return range;
}

Range.prototype.toArray = function () {
return [this.startContainer, this.startOffset, this.endContainer, this.endOffset];
}

var p1 = document.getElementById("p1");
var p2 = document.getElementById("p2");
var p3 = document.getElementById("p3");

var firstThe = [p1.firstChild, 0, p1.firstChild, 3];
var secondThe = [p1.firstChild, 11, p1.firstChild, 14];
var upperThe = [p1.firstChild, 24, p1.firstChild, 27];
var chineseMatch = [p2.firstChild, 4, p2.firstChild, 6];
var germanMatch = [p3.firstChild, 5, p3.firstChild, 6];
var rtlMatch = [p3.firstChild, 0, p3.firstChild, 3];

// When the target position is correct, at the beginning of the matched range.
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), \'the\', 11).toArray()', 'secondThe');

// Cases that the matched range is on either side of the target position.
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), \'the\', 9).toArray()', 'secondThe');
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), \'the\', 6).toArray()', 'firstThe');

// In this case, target position is at the beginning of "oog", make sure it's picking the
// left side "the" as the closest match.
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), \'the\', 16).toArray()', 'secondThe');

// Upper case.
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p1), \'The\', 11).toArray()', 'upperThe');

// CJK
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p2), \'中文\', 5).toArray()', 'chineseMatch');

// RtL
shouldBe('internals.rangeOfStringNearLocation(rangeOfNodeContent(p3), \'the\', 5).toArray()', 'rtlMatch');

container.style.display = 'none';
}

</script>
<script src="../../resources/js-test-post.js"></script>
</body>
</html>
@@ -1,3 +1,35 @@
2017-01-29 Nan Wang <n_wang@apple.com>

AX: WKContentView needs to implement UITextInput methods to make speak selection highlighting work
https://bugs.webkit.org/show_bug.cgi?id=166955

Reviewed by Ryosuke Niwa.

Created a new version of Range::collectSelectionRect() that returns rects for each
line, so that Speak Selection doesn't need to handle searching for soft line breaks.
Also added a variant of findPlainText to search for the closest matched range to the given position.

Test: editing/text-iterator/range-of-string-closest-to-position.html

* dom/Range.cpp:
(WebCore::Range::collectSelectionRectsWithoutUnionInteriorLines):
(WebCore::Range::collectSelectionRects):
* dom/Range.h:
* editing/TextIterator.cpp:
(WebCore::findPlainTextMatches):
(WebCore::updateSearchBuffer):
(WebCore::findIteratorOptions):
(WebCore::rangeMatches):
(WebCore::findClosestPlainText):
(WebCore::findPlainText):
(WebCore::findPlainTextOffset): Deleted.
* editing/TextIterator.h:
* editing/htmlediting.h:
* testing/Internals.cpp:
(WebCore::Internals::rangeOfStringNearLocation):
* testing/Internals.h:
* testing/Internals.idl:

2017-01-29 Andy Estes <aestes@apple.com>

[QuickLook] Add a WebPreference to enable saving QuickLook documents in WebKitLegacy
@@ -1266,7 +1266,7 @@ static SelectionRect coalesceSelectionRects(const SelectionRect& original, const

// This function is similar in spirit to addLineBoxRects, but annotates the returned rectangles
// with additional state which helps iOS draw selections in its unique way.
void Range::collectSelectionRects(Vector<SelectionRect>& rects)
int Range::collectSelectionRectsWithoutUnionInteriorLines(Vector<SelectionRect>& rects)
{
auto& startContainer = this->startContainer();
auto& endContainer = this->endContainer();
@@ -1325,7 +1325,7 @@ void Range::collectSelectionRects(Vector<SelectionRect>& rects)
VisiblePosition endPosition(createLegacyEditingPosition(&endContainer, endOffset), VP_DEFAULT_AFFINITY);
VisiblePosition brPosition(createLegacyEditingPosition(stopNode, 0), VP_DEFAULT_AFFINITY);
if (endPosition == brPosition)
rects.last().setIsLineBreak(true);
rects.last().setIsLineBreak(true);
}

int lineTop = std::numeric_limits<int>::max();
@@ -1422,7 +1422,15 @@ void Range::collectSelectionRects(Vector<SelectionRect>& rects)
} else if (selectionRect.direction() == LTR && selectionRect.isLastOnLine())
selectionRect.setLogicalWidth(selectionRect.maxX() - selectionRect.logicalLeft());
}

return maxLineNumber;
}

void Range::collectSelectionRects(Vector<SelectionRect>& rects)
{
int maxLineNumber = collectSelectionRectsWithoutUnionInteriorLines(rects);
const size_t numberOfRects = rects.size();

// Union all the rectangles on interior lines (i.e. not first or last).
// On first and last lines, just avoid having overlaps by merging intersecting rectangles.
Vector<SelectionRect> unionedRects;
@@ -125,6 +125,7 @@ class Range : public RefCounted<Range> {
WEBCORE_EXPORT FloatRect absoluteBoundingRect() const;
#if PLATFORM(IOS)
WEBCORE_EXPORT void collectSelectionRects(Vector<SelectionRect>&);
WEBCORE_EXPORT int collectSelectionRectsWithoutUnionInteriorLines(Vector<SelectionRect>&);
#endif

void nodeChildrenChanged(ContainerNode&);
@@ -2656,10 +2656,28 @@ static Ref<Range> collapsedToBoundary(const Range& range, bool forward)
return result;
}

static std::optional<std::pair<size_t, size_t>> findPlainTextOffset(SearchBuffer& buffer, CharacterIterator& findIterator, bool searchForward)
static TextIteratorBehavior findIteratorOptions(FindOptions options)
{
size_t matchStart = 0;
size_t matchLength = 0;
TextIteratorBehavior iteratorOptions = TextIteratorEntersTextControls | TextIteratorClipsToFrameAncestors;
if (!(options & DoNotTraverseFlatTree))
iteratorOptions |= TextIteratorTraversesFlatTree;
return iteratorOptions;
}

static void findPlainTextMatches(const Range& range, const String& target, FindOptions options, const std::function<bool(size_t, size_t)>& match)
{
SearchBuffer buffer(target, options);
if (buffer.needsMoreContext()) {
Ref<Range> beforeStartRange = range.ownerDocument().createRange();
beforeStartRange->setEnd(range.startContainer(), range.startOffset());
for (SimplifiedBackwardsTextIterator backwardsIterator(beforeStartRange.get()); !backwardsIterator.atEnd(); backwardsIterator.advance()) {
buffer.prependContext(backwardsIterator.text());
if (!buffer.needsMoreContext())
break;
}
}

CharacterIterator findIterator(range, findIteratorOptions(options));
while (!findIterator.atEnd()) {
findIterator.advance(buffer.append(findIterator.text()));
while (1) {
@@ -2674,45 +2692,53 @@ static std::optional<std::pair<size_t, size_t>> findPlainTextOffset(SearchBuffer
}
size_t lastCharacterInBufferOffset = findIterator.characterOffset();
ASSERT(lastCharacterInBufferOffset >= matchStartOffset);
matchStart = lastCharacterInBufferOffset - matchStartOffset;
matchLength = newMatchLength;
if (searchForward) // Look for the last match when searching backwards instead.
return std::pair<size_t, size_t> { matchStart, matchLength };
if (match(lastCharacterInBufferOffset - matchStartOffset, newMatchLength))
return;
}
}
}

static Ref<Range> rangeForMatch(const Range& range, FindOptions options, size_t matchStart, size_t matchLength, bool searchForward)
{
if (!matchLength)
return std::nullopt;

return std::pair<size_t, size_t> { matchStart, matchLength };
return collapsedToBoundary(range, searchForward);
CharacterIterator rangeComputeIterator(range, findIteratorOptions(options));
return characterSubrange(range.ownerDocument(), rangeComputeIterator, matchStart, matchLength);
}

Ref<Range> findPlainText(const Range& range, const String& target, FindOptions options)
Ref<Range> findClosestPlainText(const Range& range, const String& target, FindOptions options, unsigned targetOffset)
{
SearchBuffer buffer(target, options);

if (buffer.needsMoreContext()) {
Ref<Range> beforeStartRange = range.ownerDocument().createRange();
beforeStartRange->setEnd(range.startContainer(), range.startOffset());
for (SimplifiedBackwardsTextIterator backwardsIterator(beforeStartRange.get()); !backwardsIterator.atEnd(); backwardsIterator.advance()) {
buffer.prependContext(backwardsIterator.text());
if (!buffer.needsMoreContext())
break;
size_t matchStart = 0;
size_t matchLength = 0;
size_t distance = std::numeric_limits<size_t>::max();
auto match = [targetOffset, &distance, &matchStart, &matchLength] (size_t start, size_t length) {
size_t newDistance = std::min(abs(static_cast<signed>(start - targetOffset)), abs(static_cast<signed>(start + length - targetOffset)));
if (newDistance < distance) {
matchStart = start;
matchLength = length;
distance = newDistance;
}
}
return false;
};

bool searchForward = !(options & Backwards);
TextIteratorBehavior iteratorOptions = TextIteratorEntersTextControls | TextIteratorClipsToFrameAncestors;
if (!(options & DoNotTraverseFlatTree))
iteratorOptions |= TextIteratorTraversesFlatTree;
findPlainTextMatches(range, target, options, match);
return rangeForMatch(range, options, matchStart, matchLength, !(options & Backwards));
}

CharacterIterator findIterator(range, iteratorOptions);
auto result = findPlainTextOffset(buffer, findIterator, searchForward);
if (!result)
return collapsedToBoundary(range, searchForward);
Ref<Range> findPlainText(const Range& range, const String& target, FindOptions options)
{
bool searchForward = !(options & Backwards);
size_t matchStart = 0;
size_t matchLength = 0;
auto match = [searchForward, &matchStart, &matchLength] (size_t start, size_t length) {
matchStart = start;
matchLength = length;
// Look for the last match when searching backwards instead.
return searchForward;
};

CharacterIterator rangeComputeIterator(range, iteratorOptions);
return characterSubrange(range.ownerDocument(), rangeComputeIterator, result->first, result->second);
findPlainTextMatches(range, target, options, match);
return rangeForMatch(range, options, matchStart, matchLength, searchForward);
}

}
@@ -45,6 +45,7 @@ class RunResolver;
WEBCORE_EXPORT String plainText(const Range*, TextIteratorBehavior = TextIteratorDefaultBehavior, bool isDisplayString = false);
WEBCORE_EXPORT String plainTextReplacingNoBreakSpace(const Range*, TextIteratorBehavior = TextIteratorDefaultBehavior, bool isDisplayString = false);
Ref<Range> findPlainText(const Range&, const String&, FindOptions);
WEBCORE_EXPORT Ref<Range> findClosestPlainText(const Range&, const String&, FindOptions, unsigned);

// FIXME: Move this somewhere else in the editing directory. It doesn't belong here.
bool isRendererReplacedElement(RenderObject*);
@@ -144,9 +144,9 @@ bool lineBreakExistsAtVisiblePosition(const VisiblePosition&);

int comparePositions(const VisiblePosition&, const VisiblePosition&);

int indexForVisiblePosition(const VisiblePosition&, RefPtr<ContainerNode>& scope);
WEBCORE_EXPORT int indexForVisiblePosition(const VisiblePosition&, RefPtr<ContainerNode>& scope);
int indexForVisiblePosition(Node&, const VisiblePosition&, bool forSelectionPreservation);
VisiblePosition visiblePositionForIndex(int index, ContainerNode* scope);
WEBCORE_EXPORT VisiblePosition visiblePositionForIndex(int index, ContainerNode* scope);
VisiblePosition visiblePositionForIndexUsingCharacterIterator(Node&, int index); // FIXME: Why do we need this version?

// -------------------------------------------------------------------------
@@ -1456,6 +1456,11 @@ Ref<Range> Internals::subrange(Range& range, int rangeLocation, int rangeLength)
return TextIterator::subrange(&range, rangeLocation, rangeLength);
}

RefPtr<Range> Internals::rangeOfStringNearLocation(const Range& searchRange, const String& text, unsigned targetOffset)
{
return findClosestPlainText(searchRange, text, 0, targetOffset);
}

ExceptionOr<RefPtr<Range>> Internals::rangeForDictionaryLookupAtLocation(int x, int y)
{
#if PLATFORM(MAC)
@@ -201,6 +201,7 @@ class Internals final : public RefCounted<Internals>, private ContextDestruction
String rangeAsText(const Range&);
Ref<Range> subrange(Range&, int rangeLocation, int rangeLength);
ExceptionOr<RefPtr<Range>> rangeForDictionaryLookupAtLocation(int x, int y);
RefPtr<Range> rangeOfStringNearLocation(const Range&, const String&, unsigned);

ExceptionOr<void> setDelegatesScrolling(bool enabled);

@@ -195,6 +195,7 @@ enum EventThrottlingBehavior {
DOMString rangeAsText(Range range);
Range subrange(Range range, long rangeLocation, long rangeLength);
[MayThrowException] Range? rangeForDictionaryLookupAtLocation(long x, long y);
Range? rangeOfStringNearLocation(Range range, DOMString text, long targetOffset);

[MayThrowException] void setDelegatesScrolling(boolean enabled);

@@ -1,3 +1,34 @@
2017-01-29 Nan Wang <n_wang@apple.com>

AX: WKContentView needs to implement UITextInput methods to make speak selection highlighting work
https://bugs.webkit.org/show_bug.cgi?id=166955

Reviewed by Ryosuke Niwa.

Implemented methods that Speak Selection can use to retrieve the word/sentence highlighting rects.

* Scripts/webkit/messages.py:
(headers_for_type):
* UIProcess/WebPageProxy.h:
* UIProcess/WebPageProxy.messages.in:
* UIProcess/ios/WKContentViewInteraction.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView webSelectionRectsForSelectionRects:]):
(-[WKContentView webSelectionRects]):
(-[WKContentView _accessibilityRetrieveRectsEnclosingSelectionOffset:withGranularity:]):
(-[WKContentView _accessibilityRetrieveRectsAtSelectionOffset:withText:]):
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::selectionRectsCallback):
(WebKit::WebPageProxy::requestRectsForGranularityWithSelectionOffset):
(WebKit::WebPageProxy::requestRectsAtSelectionOffsetWithText):
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::visiblePositionForPositionWithOffset):
(WebKit::WebPage::getRectsForGranularityWithSelectionOffset):
(WebKit::rangeNearPositionMatchesText):
(WebKit::WebPage::getRectsAtSelectionOffsetWithText):

2017-01-29 Dan Bernstein <mitz@apple.com>

[iOS] Expose WebCore::DataDetection::detectContentInRange WKWebProcessPlugInRangeHandle
@@ -365,6 +365,7 @@ def headers_for_type(type):
'WebCore::TextIndicatorData': ['<WebCore/TextIndicator.h>'],
'WebCore::TextureMapperAnimations': ['<WebCore/TextureMapperAnimation.h>'],
'WebCore::ViewportAttributes': ['<WebCore/ViewportArguments.h>'],
'WebCore::SelectionRect': ['"EditorState.h"'],
'WebKit::BackForwardListItemState': ['"SessionState.h"'],
'WebKit::LayerHostingMode': ['"LayerTreeContext.h"'],
'WebKit::PageState': ['"SessionState.h"'],

0 comments on commit be98853

Please sign in to comment.