Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash with CJK keyboard with emoji at end of text field #42540

Merged
merged 4 commits into from Jun 12, 2023

Conversation

moffatman
Copy link
Contributor

@moffatman moffatman commented Jun 3, 2023

The isRTLAtPosition method had a bug, it used NSInteger max = [_selectionRects count] instead of NSInteger max = [_selectionRects count] - 1. But I realized we don't even need the function any more, it was used in a few places in previous iterations of #36643, but in the only place remaining, we actually already have the selection rect and don't need to search for it by position.

Btw as an explanation of the crash, I guess there is some mismatch between code point and character count somewhere. UIKit was asking for caretRectForPosition:2 when we only had 1 character. This could have only crashed when floating cursor selection was used, but actually when switching to CJK keyboard, UIKit turns out to use caretRectForPosition to calculate something about the composing rect.

Fixes flutter/flutter#128031

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides].
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See [testing the engine] for instructions on writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the [CLA].
  • All existing and new tests are passing.

It had a bug in the binary search causing a crash. But aside from that it isn't actually needed.
@cyanglaz cyanglaz requested review from LongCatIsLooong and removed request for cyanglaz June 8, 2023 18:14
@@ -1699,7 +1680,8 @@ - (CGRect)caretRectForPosition:(UITextPosition*)position {
CGRect characterAfterCaret = rects[0].rect;
// Return a zero-width rectangle along the upstream edge of the character after the caret
// position.
if ([self isRTLAtPosition:index]) {
if ([rects[0] isKindOfClass:[FlutterTextSelectionRect class]] &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old implementation doesn't seem to have this check?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also you might have to special case it when characterAfterCaret is a newline character, iirc SkParagraph reports it as an empty textbox at the end of the current line (instead of at the beginning of the next line).

Copy link
Contributor Author

@moffatman moffatman Jun 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we use selectionRectsForRange instead of directly looking at the array. I put a check, because I guess if someone subclasses and reimplemented selectionRectsForRange, this would crash if it didn't return specifically FlutterTextSelectionRect. Not sure if this is really a concern...

Not sure what you mean about the special case. But for end-of-line, it does work correctly because I have added affinity into the FlutterTextPosition. So it will go to the characterBeforeCaret section.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the document is "{newline}A" (so 2 characters), placing the cursor at (1, upstream) should point to the start of the second line. I think it would fall into the "remaining cases" category (around line 1718) and returns

    return CGRectMake(characterBeforeCaret.origin.x + characterBeforeCaret.size.width,
                      characterBeforeCaret.origin.y, 0, characterBeforeCaret.size.height);

which is going to be the right edge of the zero-width TextBox skparagraph returns for the newline character at the first line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand upstream/downstream, it only comes into play where you have word wrap breaking lines. When you have a hard line-break, we have distinct indices at the end of line 1 and the beginning of line 2. And so most of the time the affinity is downstream, exccept at the end of a word-wrapped line.

I tried out the "\nA" scenario and it seems to work as expected in all 3 possible cursor locations. In the specific one you highlighted (cursor at index 1), it used the middle branch (rects.count == 2 && affinity == UITextStorageDirectionForward). I think this isn't a concern as you will never actually get that affinity (1, upstream) using either keyboard or touch?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you will never actually get that affinity (1, upstream) using either keyboard or touch?

Ah that's probably true.

But I think newlines are still special (sorry my example was bad). I tried getting the text boxes for "A\n":

    final TextPainter painter = TextPainter(
      textDirection: TextDirection.ltr,
      text: const TextSpan(text: 'A\n'),
    )..layout();
    for (int i = 0; i < 2; i++) {
      print(painter.getBoxesForSelection(TextSelection(baseOffset: i, extentOffset: i + 1)));
    }

the output:

[TextBox.fromLTRBD(0.0, 0.0, 14.0, 14.0, TextDirection.ltr)]
[TextBox.fromLTRBD(14.0, 0.0, 14.0, 14.0, TextDirection.ltr)]

when index == 1 and the affinity is UITextStorageDirectionForward this should place the cursor at the start of the second line. But if you look at the output both TextBoxes are on the first line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I get it now. But I don't know if we can fix that here, if we don't even have any text box on the second line, we can't even make a guess of the correct position? It needs to be fixed upstream in skparagraph. And there isn't any regression in this PR. I hope we can ignore this, and merge it to avoid the hard crash.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. Probably not a SkParagraph issue I think, we should probably add a bit more information to rects the framework sends.

@@ -1727,7 +1710,8 @@ - (CGRect)caretRectForPosition:(UITextPosition*)position {
// For both cases, return a zero-width rectangle along the downstream edge of the character
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only send the rects for visible characters in a multiline text field, so technically this also includes the case when the selection reaches the end of the visible text? Would this prevent the user from scrolling the text field using force touch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is used by UIKit to figure out where to initially position its internal tracking of the selection gesture. So it would only be called on visible characters. And the logic still holds even if there are characters after current position that don't have a rect. As long as we have the 1 visible character before the cursor.
And yeah, to implement full scrolling we would need to send rects for all characters. A matter for another PR in framework.

@@ -1727,7 +1710,8 @@ - (CGRect)caretRectForPosition:(UITextPosition*)position {
// For both cases, return a zero-width rectangle along the downstream edge of the character
// before the caret position.
CGRect characterBeforeCaret = rects[0].rect;
if ([self isRTLAtPosition:index - 1]) {
if ([rects[0] isKindOfClass:[FlutterTextSelectionRect class]] &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can assert the class type here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean. I am checking the class type?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

FlutterTextSelectionRect *rect = (FlutterTextSelectionRect *)rects[0];
if (rect...)

Because the rect must be the correct type right? There is no way isKindOfClass returns false, am I right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically it's possible, this method signature is not FlutterTextSelectionRect*, but UITextSelectionRect*.

@moffatman moffatman added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 12, 2023
@auto-submit auto-submit bot merged commit de68fba into flutter:main Jun 12, 2023
27 checks passed
auto-submit bot pushed a commit to flutter/flutter that referenced this pull request Jun 12, 2023
…128721)

flutter/engine@33e0693...de68fba

2023-06-12 smartercallum@gmail.com Fix crash with CJK keyboard with emoji at end of text field (flutter/engine#42540)
2023-06-12 skia-flutter-autoroll@skia.org Roll Skia from 658b1d366758 to 6bdb0ef30cb6 (2 revisions) (flutter/engine#42778)
2023-06-12 jonahwilliams@google.com [Impeller] Correct attachment description for offscreen MSAA resolve. (flutter/engine#42753)
2023-06-12 tamird@google.com Remove dependency on memfs (flutter/engine#42773)
2023-06-12 skia-flutter-autoroll@skia.org Roll Skia from 0f974a0f8c10 to 658b1d366758 (1 revision) (flutter/engine#42776)
2023-06-12 skia-flutter-autoroll@skia.org Roll ANGLE from 3abbc4f99970 to 43ef50f389e9 (1 revision) (flutter/engine#42775)
2023-06-12 kjlubick@users.noreply.github.com Remove unnecessary #include of SkPromiseImageTexture (flutter/engine#42770)
2023-06-12 skia-flutter-autoroll@skia.org Roll Skia from 951123096e55 to 0f974a0f8c10 (5 revisions) (flutter/engine#42771)
2023-06-12 jonahwilliams@google.com [Impeller] opt all vertex shader position/uvs into highp (flutter/engine#42746)
2023-06-12 30870216+gaaclarke@users.noreply.github.com [Impeller] added debug info to frame debuggers like AGI (flutter/engine#42717)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jacksongardner@google.com,rmistry@google.com,zra@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Jasguerrero pushed a commit to Jasguerrero/engine that referenced this pull request Jun 17, 2023
…42540)

The `isRTLAtPosition` method had a bug, it used `NSInteger max = [_selectionRects count]` instead of `NSInteger max = [_selectionRects count] - 1`. But I realized we don't even need the function any more, it was used in a few places in previous iterations of flutter#36643, but in the only place remaining, we actually already have the selection rect and don't need to search for it by position.

Btw as an explanation of the crash, I guess there is some mismatch between code point and character count somewhere. UIKit was asking for `caretRectForPosition:2` when we only had 1 character. This could have only crashed when floating cursor selection was used, but actually when switching to CJK keyboard, UIKit turns out to use `caretRectForPosition` to calculate something about the composing rect.

Fixes flutter/flutter#128031
auto-submit bot pushed a commit that referenced this pull request Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit Merge PR when tree becomes green via auto submit App platform-ios
Projects
None yet
3 participants