Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let apps open and close the keyboard on demand, and let selection exist when keyboard is closed (Resolves #875) #876

Merged
merged 27 commits into from
Jan 10, 2023

Conversation

matthew-carroll
Copy link
Contributor

This PR lets apps open and close the IME through a SuperEditor DocumentComposer. It also lets the document selection exist even after the keyboard has closed.

@matthew-carroll
Copy link
Contributor Author

@Jethro87 can you give this PR a try for your keyboard use-case?

I made a bit of a mess of the IME code. But before I spend time cleaning it up, I want to make sure we solved the interactions that you need.

To see a demo of the behavior that you're interested in, check the example app for lib/main_panel_behind_keyboard.dart

Here's a video:

Screen.Recording.2022-12-04.at.11.39.38.PM.mov

@Jethro87
Copy link
Contributor

Jethro87 commented Dec 5, 2022

@matthew-carroll The demo interactions work as expected. I integrated the "above keyboard" mobile toolbar into the "expanded" toolbar to test if adding attributions to the selected text works, and it worked well.

Whenever I type anything, however, I get this exception:

══╡ EXCEPTION CAUGHT BY SERVICES LIBRARY ╞══════════════════════════════════════════════════════════
The following _CastError was thrown during method call TextInputClient.updateEditingStateWithDeltas:
Null check operator used on a null value

When the exception was thrown, this was the stack:
#0      EditorImeClient.updateEditingValueWithDeltas (package:super_editor/src/core/document_ime.dart:111:23)
package:super_editor/src/core/document_ime.dart:111
#1      TextInput._handleTextInputInvocation (package:flutter/src/services/text_input.dart:1867:63)
package:flutter/src/services/text_input.dart:1867
#2      TextInput._loudlyHandleTextInputInvocation (package:flutter/src/services/text_input.dart:1753:20)
package:flutter/src/services/text_input.dart:1753
#3      MethodChannel._handleAsMethodCall (package:flutter/src/services/platform_channel.dart:555:55)
package:flutter/src/services/platform_channel.dart:555
#4      MethodChannel.setMethodCallHandler.<anonymous closure> (package:flutter/src/services/platform_channel.dart:548:34)
package:flutter/src/services/platform_channel.dart:548
#5      _DefaultBinaryMessenger.setMessageHandler.<anonymous closure> (package:flutter/src/services/binding.dart:393:35)
package:flutter/src/services/binding.dart:393
#6      _invoke2 (dart:ui/hooks.dart:183:13)
dart:ui/hooks.dart:183
#7      _ChannelCallbackRecord.invoke (dart:ui/channel_buffers.dart:40:5)
dart:ui/channel_buffers.dart:40
#8      _Channel.push (dart:ui/channel_buffers.dart:130:31)
dart:ui/channel_buffers.dart:130
#9      ChannelBuffers.push (dart:ui/channel_buffers.dart:326:17)
dart:ui/channel_buffers.dart:326
#10     PlatformDispatcher._dispatchPlatformMessage (dart:ui/platform_dispatcher.dart:664:22)
dart:ui/platform_dispatcher.dart:664
#11     _dispatchPlatformMessage (dart:ui/hooks.dart:86:31)
dart:ui/hooks.dart:86

call:
  MethodCall(TextInputClient.updateEditingStateWithDeltas, [1, {deltas: [{oldText: , deltaText: ,
  deltaStart: -1, deltaEnd: -1, selectionBase: 0, selectionExtent: 0, composingBase: -1,
  composingExtent: -1}, {oldText: , deltaText: H, deltaStart: 0, deltaEnd: 0, selectionBase: 1,
  selectionExtent: 1, composingBase: 0, composingExtent: 1}]}])
════════════════════════════════════════════════════════════════════════════════════════════════════
Another exception was thrown: Null check operator used on a null value
Another exception was thrown: Null check operator used on a null value
Another exception was thrown: Null check operator used on a null value
Another exception was thrown: Null check operator used on a null value

@matthew-carroll
Copy link
Contributor Author

@Jethro87 do you wanna try this again and see if that bug is resolved?

If it is, do you want to try integrating this on a branch of your app and see how it goes? There will be some further massaging of the API and implementation, but if you try to integrate the current implementation, it might demonstrate the parts that work well, and the parts that don't.

@Jethro87
Copy link
Contributor

Jethro87 commented Dec 6, 2022

@matthew-carroll The typing bug has been resolved, but I found a couple more bugs. I will list them below. In the meantime, I'll integrate this into Clearful and let you know how it goes.

The following bugs occur with this base document:

MutableDocument(nodes: [
      ParagraphNode(
        id: DocumentEditor.createNodeId(),
        text: AttributedText(),
      ),
      ParagraphNode(
        id: DocumentEditor.createNodeId(),
        text: AttributedText(),
      ),
      ParagraphNode(
        id: DocumentEditor.createNodeId(),
        text: AttributedText(text: 'Friend'),
      ),
      ParagraphNode(
        id: DocumentEditor.createNodeId(),
        text: AttributedText(text: 'Hello'),
      ),
    ]);

They both occur when grabbing the single selection handle (Android) and moving it anywhere in the document.

Bug 1: Move the selection handle anywhere in the document without typing first

══╡ EXCEPTION CAUGHT BY SERVICES LIBRARY ╞══════════════════════════════════════════════════════════
The following StateError was thrown during method call TextInputClient.updateEditingStateWithDeltas:
Bad state: No element

When the exception was thrown, this was the stack:
#0      List.last (dart:core-patch/growable_array.dart:348:5)
dart:core-patch/growable_array.dart:348
#1      EditorImeClient.updateEditingValueWithDeltas (package:super_editor/src/core/document_ime.dart:79:41)
package:super_editor/src/core/document_ime.dart:79
#2      TextInput._handleTextInputInvocation (package:flutter/src/services/text_input.dart:1867:63)
package:flutter/src/services/text_input.dart:1867
#3      TextInput._loudlyHandleTextInputInvocation (package:flutter/src/services/text_input.dart:1753:20)
package:flutter/src/services/text_input.dart:1753
#4      MethodChannel._handleAsMethodCall (package:flutter/src/services/platform_channel.dart:555:55)
package:flutter/src/services/platform_channel.dart:555
#5      MethodChannel.setMethodCallHandler.<anonymous closure> (package:flutter/src/services/platform_channel.dart:548:34)
package:flutter/src/services/platform_channel.dart:548
#6      _DefaultBinaryMessenger.setMessageHandler.<anonymous closure> (package:flutter/src/services/binding.dart:393:35)
package:flutter/src/services/binding.dart:393
#7      _invoke2 (dart:ui/hooks.dart:183:13)
dart:ui/hooks.dart:183
#8      _ChannelCallbackRecord.invoke (dart:ui/channel_buffers.dart:40:5)
dart:ui/channel_buffers.dart:40
#9      _Channel.push (dart:ui/channel_buffers.dart:130:31)
dart:ui/channel_buffers.dart:130
#10     ChannelBuffers.push (dart:ui/channel_buffers.dart:326:17)
dart:ui/channel_buffers.dart:326
#11     PlatformDispatcher._dispatchPlatformMessage (dart:ui/platform_dispatcher.dart:664:22)
dart:ui/platform_dispatcher.dart:664
#12     _dispatchPlatformMessage (dart:ui/hooks.dart:86:31)
dart:ui/hooks.dart:86

call:
  MethodCall(TextInputClient.updateEditingStateWithDeltas, [1, {deltas: []}])
════════════════════════════════════════════════════════════════════════════════════════════════════

Bug 2: Type anything in the document, then move the selection handle anywhere else in the document

══╡ EXCEPTION CAUGHT BY FOUNDATION LIBRARY ╞════════════════════════════════════════════════════════
The following assertion was thrown while dispatching notifications for
ValueNotifier<DocumentSelection?>:
Range end 9 is out of text of length 6
'package:flutter/src/services/text_input.dart':
Failed assertion: line 973 pos 12: 'range.end >= 0 && range.end <= text.length'

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  https://github.com/flutter/flutter/issues/new?template=2_bug.md

When the exception was thrown, this was the stack:
#2      TextEditingValue._textRangeIsValid (package:flutter/src/services/text_input.dart:973:12)
package:flutter/src/services/text_input.dart:973
#3      TextEditingValue.toJSON (package:flutter/src/services/text_input.dart:927:12)
package:flutter/src/services/text_input.dart:927
#4      _PlatformTextInputControl.setEditingState (package:flutter/src/services/text_input.dart:2267:13)
package:flutter/src/services/text_input.dart:2267
#5      TextInput._setEditingState (package:flutter/src/services/text_input.dart:1955:15)
package:flutter/src/services/text_input.dart:1955
#6      TextInputConnection.setEditingState (package:flutter/src/services/text_input.dart:1361:25)
package:flutter/src/services/text_input.dart:1361
#7      EditorImeClient.currentTextEditingValue= (package:super_editor/src/core/document_ime.dart:46:22)
package:super_editor/src/core/document_ime.dart:46
#8      DocumentComposer._syncImeWithDocumentAndSelection (package:super_editor/src/core/document_composer.dart:222:18)
package:super_editor/src/core/document_composer.dart:222
#9      DocumentComposer.showImeInput (package:super_editor/src/core/document_composer.dart:171:7)
package:super_editor/src/core/document_composer.dart:171
#10     _DocumentImeInteractorState._onComposerChange (package:super_editor/src/default_editor/document_input_ime.dart:159:35)
package:super_editor/src/default_editor/document_input_ime.dart:159
#11     ChangeNotifier.notifyListeners (package:flutter/src/foundation/change_notifier.dart:381:24)
package:flutter/src/foundation/change_notifier.dart:381
#12     ValueNotifier.value= (package:flutter/src/foundation/change_notifier.dart:495:5)
package:flutter/src/foundation/change_notifier.dart:495
#13     _AndroidDocumentTouchInteractorState._updateSelectionForNewDragHandleLocation (package:super_editor/src/default_editor/document_gestures_touch_android.dart:677:24)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:677
#14     _AndroidDocumentTouchInteractorState._onHandleDragUpdate (package:super_editor/src/default_editor/document_gestures_touch_android.dart:657:5)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:657
#15     _AndroidDocumentTouchEditingControlsState._onPanUpdate (package:super_editor/src/default_editor/document_gestures_touch_android.dart:1115:32)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:1115
#16     DragGestureRecognizer._checkUpdate.<anonymous closure> (package:flutter/src/gestures/monodrag.dart:483:55)
package:flutter/src/gestures/monodrag.dart:483
#17     GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:253:24)
package:flutter/src/gestures/recognizer.dart:253
#18     DragGestureRecognizer._checkUpdate (package:flutter/src/gestures/monodrag.dart:483:7)
package:flutter/src/gestures/monodrag.dart:483
#19     DragGestureRecognizer.handleEvent (package:flutter/src/gestures/monodrag.dart:330:9)
package:flutter/src/gestures/monodrag.dart:330
#20     PointerRouter._dispatch (package:flutter/src/gestures/pointer_router.dart:98:12)
package:flutter/src/gestures/pointer_router.dart:98
#21     PointerRouter._dispatchEventToRoutes.<anonymous closure> (package:flutter/src/gestures/pointer_router.dart:143:9)
package:flutter/src/gestures/pointer_router.dart:143
#22     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:625:13)
dart:collection-patch/compact_hash.dart:625
#23     PointerRouter._dispatchEventToRoutes (package:flutter/src/gestures/pointer_router.dart:141:18)
package:flutter/src/gestures/pointer_router.dart:141
#24     PointerRouter.route (package:flutter/src/gestures/pointer_router.dart:127:7)
package:flutter/src/gestures/pointer_router.dart:127
#25     GestureBinding.handleEvent (package:flutter/src/gestures/binding.dart:460:19)
package:flutter/src/gestures/binding.dart:460
#26     GestureBinding.dispatchEvent (package:flutter/src/gestures/binding.dart:440:22)
package:flutter/src/gestures/binding.dart:440
#27     RendererBinding.dispatchEvent (package:flutter/src/rendering/binding.dart:336:11)
package:flutter/src/rendering/binding.dart:336
#28     GestureBinding._handlePointerEventImmediately (package:flutter/src/gestures/binding.dart:395:7)
package:flutter/src/gestures/binding.dart:395
#29     GestureBinding.handlePointerEvent (package:flutter/src/gestures/binding.dart:357:5)
package:flutter/src/gestures/binding.dart:357
#30     GestureBinding._flushPointerEventQueue (package:flutter/src/gestures/binding.dart:314:7)
package:flutter/src/gestures/binding.dart:314
#31     GestureBinding._handlePointerDataPacket (package:flutter/src/gestures/binding.dart:295:7)
package:flutter/src/gestures/binding.dart:295
#32     _invoke1 (dart:ui/hooks.dart:164:13)
dart:ui/hooks.dart:164
#33     PlatformDispatcher._dispatchPointerDataPacket (dart:ui/platform_dispatcher.dart:361:7)
dart:ui/platform_dispatcher.dart:361
#34     _dispatchPointerDataPacket (dart:ui/hooks.dart:91:31)
dart:ui/hooks.dart:91
(elided 2 frames from class _AssertionError)
/(elided:2

The ValueNotifier<DocumentSelection?> sending notification was:
  ValueNotifier<DocumentSelection?>#fbce2([DocumentSelection] -
    base: ([DocumentPosition] - node: "bd3cdc42-c2b7-491d-9d9c-d5bc59c0fe76", position:
  (TextPosition(offset: 7, affinity: TextAffinity.downstream))),
    extent: ([DocumentPosition] - node: "bd3cdc42-c2b7-491d-9d9c-d5bc59c0fe76", position:
  (TextPosition(offset: 7, affinity: TextAffinity.downstream))))
════════════════════════════════════════════════════════════════════════════════════════════════════

@matthew-carroll
Copy link
Contributor Author

@Jethro87 I fixed at least one composing region issue. Can you try again?

@Jethro87
Copy link
Contributor

Jethro87 commented Dec 7, 2022

@matthew-carroll Got another assertion error. This happened again after typing in a doc, then moving the selection handle around.

══╡ EXCEPTION CAUGHT BY FOUNDATION LIBRARY ╞════════════════════════════════════════════════════════
The following assertion was thrown while dispatching notifications for
ValueNotifier<DocumentSelection?>:
Range end 8 is out of text of length 5
'package:flutter/src/services/text_input.dart':
Failed assertion: line 973 pos 12: 'range.end >= 0 && range.end <= text.length'

Either the assertion indicates an error in the framework itself, or we should provide substantially
more information in this error message to help you determine and fix the underlying cause.
In either case, please report this assertion by filing a bug on GitHub:
  https://github.com/flutter/flutter/issues/new?template=2_bug.md

When the exception was thrown, this was the stack:
#2      TextEditingValue._textRangeIsValid (package:flutter/src/services/text_input.dart:973:12)
package:flutter/src/services/text_input.dart:973
#3      TextEditingValue.toJSON (package:flutter/src/services/text_input.dart:927:12)
package:flutter/src/services/text_input.dart:927
#4      _PlatformTextInputControl.setEditingState (package:flutter/src/services/text_input.dart:2267:13)
package:flutter/src/services/text_input.dart:2267
#5      TextInput._setEditingState (package:flutter/src/services/text_input.dart:1955:15)
package:flutter/src/services/text_input.dart:1955
#6      TextInputConnection.setEditingState (package:flutter/src/services/text_input.dart:1361:25)
package:flutter/src/services/text_input.dart:1361
#7      EditorImeClient.currentTextEditingValue= (package:super_editor/src/core/document_ime.dart:46:22)
package:super_editor/src/core/document_ime.dart:46
#8      DocumentComposer._syncImeWithDocumentAndSelection (package:super_editor/src/core/document_composer.dart:222:18)
package:super_editor/src/core/document_composer.dart:222
#9      DocumentComposer.showImeInput (package:super_editor/src/core/document_composer.dart:171:7)
package:super_editor/src/core/document_composer.dart:171
#10     _DocumentImeInteractorState._onComposerChange (package:super_editor/src/default_editor/document_input_ime.dart:159:35)
package:super_editor/src/default_editor/document_input_ime.dart:159
#11     ChangeNotifier.notifyListeners (package:flutter/src/foundation/change_notifier.dart:381:24)
package:flutter/src/foundation/change_notifier.dart:381
#12     ValueNotifier.value= (package:flutter/src/foundation/change_notifier.dart:495:5)
package:flutter/src/foundation/change_notifier.dart:495
#13     _AndroidDocumentTouchInteractorState._updateSelectionForNewDragHandleLocation (package:super_editor/src/default_editor/document_gestures_touch_android.dart:677:24)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:677
#14     _AndroidDocumentTouchInteractorState._onHandleDragUpdate (package:super_editor/src/default_editor/document_gestures_touch_android.dart:657:5)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:657
#15     _AndroidDocumentTouchEditingControlsState._onPanUpdate (package:super_editor/src/default_editor/document_gestures_touch_android.dart:1115:32)
package:super_editor/src/default_editor/document_gestures_touch_android.dart:1115
#16     DragGestureRecognizer._checkUpdate.<anonymous closure> (package:flutter/src/gestures/monodrag.dart:483:55)
package:flutter/src/gestures/monodrag.dart:483
#17     GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:253:24)
package:flutter/src/gestures/recognizer.dart:253
#18     DragGestureRecognizer._checkUpdate (package:flutter/src/gestures/monodrag.dart:483:7)
package:flutter/src/gestures/monodrag.dart:483
#19     DragGestureRecognizer.handleEvent (package:flutter/src/gestures/monodrag.dart:330:9)
package:flutter/src/gestures/monodrag.dart:330
#20     PointerRouter._dispatch (package:flutter/src/gestures/pointer_router.dart:98:12)
package:flutter/src/gestures/pointer_router.dart:98
#21     PointerRouter._dispatchEventToRoutes.<anonymous closure> (package:flutter/src/gestures/pointer_router.dart:143:9)
package:flutter/src/gestures/pointer_router.dart:143
#22     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:625:13)
dart:collection-patch/compact_hash.dart:625
#23     PointerRouter._dispatchEventToRoutes (package:flutter/src/gestures/pointer_router.dart:141:18)
package:flutter/src/gestures/pointer_router.dart:141
#24     PointerRouter.route (package:flutter/src/gestures/pointer_router.dart:127:7)
package:flutter/src/gestures/pointer_router.dart:127
#25     GestureBinding.handleEvent (package:flutter/src/gestures/binding.dart:460:19)
package:flutter/src/gestures/binding.dart:460
#26     GestureBinding.dispatchEvent (package:flutter/src/gestures/binding.dart:440:22)
package:flutter/src/gestures/binding.dart:440
#27     RendererBinding.dispatchEvent (package:flutter/src/rendering/binding.dart:336:11)
package:flutter/src/rendering/binding.dart:336
#28     GestureBinding._handlePointerEventImmediately (package:flutter/src/gestures/binding.dart:395:7)
package:flutter/src/gestures/binding.dart:395
#29     GestureBinding.handlePointerEvent (package:flutter/src/gestures/binding.dart:357:5)
package:flutter/src/gestures/binding.dart:357
#30     GestureBinding._flushPointerEventQueue (package:flutter/src/gestures/binding.dart:314:7)
package:flutter/src/gestures/binding.dart:314
#31     GestureBinding._handlePointerDataPacket (package:flutter/src/gestures/binding.dart:295:7)
package:flutter/src/gestures/binding.dart:295
#32     _invoke1 (dart:ui/hooks.dart:164:13)
dart:ui/hooks.dart:164
#33     PlatformDispatcher._dispatchPointerDataPacket (dart:ui/platform_dispatcher.dart:361:7)
dart:ui/platform_dispatcher.dart:361
#34     _dispatchPointerDataPacket (dart:ui/hooks.dart:91:31)
dart:ui/hooks.dart:91
(elided 2 frames from class _AssertionError)
/(elided:2

The ValueNotifier<DocumentSelection?> sending notification was:
  ValueNotifier<DocumentSelection?>#c92c0([DocumentSelection] -
    base: ([DocumentPosition] - node: "237cac3d-7133-4630-a444-4d740e0bdc49", position:
  (TextPosition(offset: 0, affinity: TextAffinity.downstream))),
    extent: ([DocumentPosition] - node: "237cac3d-7133-4630-a444-4d740e0bdc49", position:
  (TextPosition(offset: 0, affinity: TextAffinity.downstream))))
════════════════════════════════════════════════════════════════════════════════════════════════════
Another exception was thrown: Range end 8 is out of text of length 5
Another exception was thrown: Exception: Couldn't map an IME position to a document position. IME position: TextPosition(offset: 9, affinity: TextAffinity.downstream)
Another exception was thrown: Bad state: No element

@matthew-carroll
Copy link
Contributor Author

@Jethro87 can you let me know what you did, exactly? My initial attempt to type and move the caret isn't crashing anything on my end.

@Jethro87
Copy link
Contributor

Jethro87 commented Dec 7, 2022

@matthew-carroll I will attempt to create a simpler example that can be reproduced. I've done much of the integration into my app and it's there that I've been testing this.

FWIW, this doesn't seem to be happening at all on iOS.

@Jethro87
Copy link
Contributor

Jethro87 commented Dec 8, 2022

@matthew-carroll I found a bug I can reproduce on iOS and Android. It's detailed below.

Also, I have a question. Currently, tapping the editor causes the keyboard to rise. This is expected when a document is opened and it there is no selection / the IME is not attached. I don't necessarily expect this when the editor has selection / the IME is attached and the keyboard is lowered with the panel visible. In this state, tapping the editor anywhere will bring the keyboard back up. If a user has a document that they want to format, for example, and the formatting options are in the behind-keyboard-panel, they would have to double tap to select a word, lower the keyboard, apply the format, tap another word (which raises the keyboard), lower the keyboard, apply the format, etc, etc.

In both your demo and my app there is a button on top of the keyboard/panel that allows the user to dismiss the panel and bring the keyboard back up. Now that the editor can continue to have selection when the keyboard is down, I'm wondering if the responsibility to raise the keyboard should be on the developer, rather than super_editor automatically raising the keyboard on tap.

Something like this:

  • If the editor has no selection / the IME is not attached, tapping raises the keyboard and attaches the IME
  • If the editor has selection / IME attached, tapping does not raise the keyboard

What do you think?

--

Here's how to reproduce the bug

  • Open a document and double tap to highlight a word
  • Move the selection handles around (all should work fine)
  • With the same selection highlighted, lower the keyboard
  • While the keyboard is lowered, drag the selection around again

The following happens to me reliably on iOS and Android:

══╡ EXCEPTION CAUGHT BY SCHEDULER LIBRARY ╞═════════════════════════════════════════════════════════
The following assertion was thrown during a scheduler callback:
A ScrollPositionWithSingleContext was used after being disposed.
Once you have called dispose() on a ScrollPositionWithSingleContext, it can no longer be used.

When the exception was thrown, this was the stack:
#0      ChangeNotifier.debugAssertNotDisposed.<anonymous closure> (package:flutter/src/foundation/change_notifier.dart:157:9)
package:flutter/src/foundation/change_notifier.dart:157
#1      ChangeNotifier.debugAssertNotDisposed (package:flutter/src/foundation/change_notifier.dart:164:6)
package:flutter/src/foundation/change_notifier.dart:164
#2      ChangeNotifier.notifyListeners (package:flutter/src/foundation/change_notifier.dart:360:27)
package:flutter/src/foundation/change_notifier.dart:360
#3      ScrollPosition.notifyListeners (package:flutter/src/widgets/scroll_position.dart:984:11)
package:flutter/src/widgets/scroll_position.dart:984
#4      ScrollPosition.forcePixels (package:flutter/src/widgets/scroll_position.dart:384:5)
package:flutter/src/widgets/scroll_position.dart:384
#5      ScrollPositionWithSingleContext.jumpTo (package:flutter/src/widgets/scroll_position_with_single_context.dart:202:7)
package:flutter/src/widgets/scroll_position_with_single_context.dart:202
#6      AutoScroller._scrollDown (package:super_editor/src/infrastructure/_scrolling.dart:102:22)
package:super_editor/src/infrastructure/_scrolling.dart:102
#7      AutoScroller._onTick (package:super_editor/src/infrastructure/_scrolling.dart:125:7)
package:super_editor/src/infrastructure/_scrolling.dart:125
#8      Ticker._tick (package:flutter/src/scheduler/ticker.dart:249:12)
package:flutter/src/scheduler/ticker.dart:249
#9      SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1289:15)
package:flutter/src/scheduler/binding.dart:1289
#10     SchedulerBinding.handleBeginFrame.<anonymous closure> (package:flutter/src/scheduler/binding.dart:1140:11)
package:flutter/src/scheduler/binding.dart:1140
#11     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:625:13)
dart:collection-patch/compact_hash.dart:625
#12     SchedulerBinding.handleBeginFrame (package:flutter/src/scheduler/binding.dart:1138:17)
package:flutter/src/scheduler/binding.dart:1138
#13     _invoke1 (dart:ui/hooks.dart:164:13)
dart:ui/hooks.dart:164
2

This exception was thrown in the context of a scheduler callback. When the scheduler callback was _registered_ (as opposed to when the exception was thrown), this was the stack:
#2      SchedulerBinding.scheduleFrameCallback (package:flutter/src/scheduler/binding.dart:571:49)
package:flutter/src/scheduler/binding.dart:571
#3      Ticker.scheduleTick (package:flutter/src/scheduler/ticker.dart:265:46)
package:flutter/src/scheduler/ticker.dart:265
#4      Ticker.start (package:flutter/src/scheduler/ticker.dart:171:7)
package:flutter/src/scheduler/ticker.dart:171
#5      AutoScroller.startScrollingDown (package:super_editor/src/infrastructure/_scrolling.dart:85:13)
package:super_editor/src/infrastructure/_scrolling.dart:85
#6      DragHandleAutoScroller.updateAutoScrollHandleMonitoring (package:super_editor/src/infrastructure/platforms/mobile_documents.dart:217:21)
package:super_editor/src/infrastructure/platforms/mobile_documents.dart:217
#7      _IOSDocumentTouchInteractorState._onPanUpdate (package:super_editor/src/default_editor/document_gestures_touch_ios.dart:735:26)
package:super_editor/src/default_editor/document_gestures_touch_ios.dart:735
#8      DragGestureRecognizer._checkUpdate.<anonymous closure> (package:flutter/src/gestures/monodrag.dart:483:55)
package:flutter/src/gestures/monodrag.dart:483
#9      GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:253:24)
package:flutter/src/gestures/recognizer.dart:253

@matthew-carroll
Copy link
Contributor Author

@Jethro87 I tried those steps on an emulator and a Pixel 3a, but I wasn't able to cause a crash when moving the handles with the keyboard collapsed.

I'll take a look at the keyboard expansion issue once we work out the bugs with the current implementation.

@matthew-carroll
Copy link
Contributor Author

@Jethro87 do you have more specific repro steps that we can try?

@Jethro87
Copy link
Contributor

@matthew-carroll I've been testing this quite a bit in Clearful the past few days. Things are going well overall, and I haven't been able to reproduce those selection bugs that were happening. Not sure what happened there.

I've found two issues that have been introduced by having to set resizeToAvoidBottomInset: false in the Scaffold.

1. When the editor contains a document with many nodes and the keyboard is up, I cannot scroll to the bottom of the document.

RPReplay_Final1670962858.mov

2. Auto-scrolling no longer works properly when tapping a document without selection.

RPReplay_Final1670962902.mov

@matthew-carroll
Copy link
Contributor Author

Re: can't scroll to end...

Apply viewInsets around SuperEditor or whatever subtree fills your content area:

@override
  Widget build(BuildContext context) {
    return Scaffold(
      resizeToAvoidBottomInset: false,
      body: Stack(
        children: [
          Positioned.fill(
            child: Padding(
              padding: MediaQuery.of(context).viewInsets,
              child: SuperEditor(
                editor: _editor,
                composer: _composer,
              ),
            ),
          ),
          Positioned(
            left: 0,
            right: 0,
            bottom: 0,
            child: BehindKeyboardPanel(
              onOpenKeyboard: _openKeyboard,
              onCloseKeyboard: _closeKeyboard,
              onEndEditing: _endEditing,
            ),
          ),
        ],
      ),
    );
  }

@matthew-carroll
Copy link
Contributor Author

Re: no auto-scrolling...

The above solution, which changes the height of SuperEditor should also fix the auto-scroll problem. If you have content selected that sits behind the keyboard, when the keyboard comes up, you should see SuperEditor scroll to display it.

…uperEditor doesn't flow behind keyboard panel.
@Jethro87
Copy link
Contributor

@matthew-carroll That fixed both issues. Thanks!

@matthew-carroll
Copy link
Contributor Author

Great. Is the only remaining issue the fact that re-selecting a selected document automatically raises the keyboard? Or are there still other use-case issues?

@Jethro87
Copy link
Contributor

@matthew-carroll I haven't found any other issues. Things look good, other than re-selecting a selected document automatically raises the keyboard.

@matthew-carroll
Copy link
Contributor Author

@Jethro87 let me know if the latest change successfully avoids any keyboard opening when you don't want it to.

If we've solved all the use-cases, then it will be time to add tests to lock down the behaviors, followed by a general IME refactoring to create pieces that make more sense for behavior changes like this.

@Jethro87
Copy link
Contributor

@matthew-carroll Mostly, it looks good, though I noticed that double tapping automatically dismisses the keyboard / shows the panel. We want the user to dismiss the keyboard / show the panel explicitly. Once the panel is shown, they also have to close it explicitly.

Other than this, I think the functionality we want is all there.

@matthew-carroll
Copy link
Contributor Author

@Jethro87 let me know if that fixes the double and triple tap keyboard issues.

@matthew-carroll matthew-carroll marked this pull request as ready for review January 8, 2023 07:07
@matthew-carroll
Copy link
Contributor Author

@Jethro87 I've pushed a candidate for the final API. Here's an explanation.

Previously, I tried moving a bunch of IME stuff into the DocumentComposer. It felt like that class was turning into a God object, so I reversed course.

The current approach is to create as many small widgets as possible, which take up narrow responsibilities.

At a high level, you control IME behavior with SuperEditor using widget properties:

return SuperEditor(
    // Other properties...
    softwareKeyboardController: _keyboardController,
    imePolicies: SuperEditorImePolicies(
        openKeyboardOnSelectionChange: false,
        clearSelectionWhenImeDisconnects: false,
    ),
)

Use the SoftwareKeyboardController to open and close the keyboard, as desired. Use the SuperEditorImePolicies to turn off keyboard behaviors that would otherwise happen automatically.

This PR converts the document_input_ime.dart into an entire directory of files. Here are some of the noteworthy classes:

SuperEditorImeInteractor- Owns all IME responsibility within the SuperEditor widget. This class is implemented specifically for SuperEditor behaviors.

SuperEditorHardwareKeyHandler - A renamed version of the class that was already being used for hardware key presses in physical keyboard mode. Now, that widget is used in both input modes, but in IME mode it responds to far fewer keys.

DocumentSelectionOpenAndClosePolicy - Compositional widget, which opens and closes the keyboard automatically, based on changes to the selection, and also clears the selection when the keyboard closes. This widget is configured based on the aforementioned IME policies that you pass into SuperEditor.

ImeFocusPolicy - Compositional widget, which closes the IME connection when SuperEditor loses focus.

SoftwareKeyboardOpener - Compositional widget, which opens and closes the software keyboard, as directed by a given SoftwareKeyboardController.

DocumentToImeSynchronizer - Compositional widget, which watches a document, selection, and composing region. When any of those things change, this widget serializes them into a TextEditingValue and sends them to the IME.

DocumentImeInputClient - Dart objects that receives deltas from the platform IME and forwards them to a TextDeltasDocumentEditor.

TextDeltasDocumentEditor - Dart object that changes document content based on deltas.

DocumentImeSerializer - Maps between structured documents and flat text required by the IME.

Please let me know if this API looks good to you.

@Jethro87
Copy link
Contributor

Jethro87 commented Jan 8, 2023

@matthew-carroll I've updated my repo with the proposed API and am testing it. It looks good so far.

I did notice one bug that happens whenever I pop the screen I'm on. This happens in the _closeKeyboard(); call in PanelBehindKeyboardDemo's dispose method.

flutter: Closing keyboard (and disconnecting from IME)

════════ Exception caught by widgets library ═══════════════════════════════════
The following assertion was thrown while finalizing the widget tree:
'package:super_editor/src/default_editor/document_ime/ime_keyboard_control.dart': Failed assertion: line 124 pos 12: 'hasDelegate': is not true.
package:super_editor/…/document_ime/ime_keyboard_control.dart:124

When the exception was thrown, this was the stack
#2      SoftwareKeyboardController.close
package:super_editor/…/document_ime/ime_keyboard_control.dart:124
#3      _SuperEditorDocumentState._closeKeyboard
package:clearful/…/presentation/editor.dart:144
#4      _SuperEditorDocumentState.dispose

@matthew-carroll
Copy link
Contributor Author

@Jethro87 can you let me know exactly what I need to do to trigger that error?

@Jethro87
Copy link
Contributor

Jethro87 commented Jan 8, 2023

@matthew-carroll Sorry. I have a couple of SuperEditor's in my app, both in separate routes that I navigate to. One SuperEditor is our Clearful integration, one is the stock copy/pasted PanelBehindKeyboardDemo widget in example/lib/demos/experiments/demo_panel_behind_keyboard. Upon popping the routes of either of these editors, the error appears.

I did some more digging, and I can remove the error if I wrap the closeKeyboard() call in the dispose method with a delegate check. Shown below:

  @override
  void dispose() {
    if (_keyboardController.hasDelegate) {
      _closeKeyboard();
    }
    _composer.dispose();
    _focusNode.dispose();
    super.dispose();
  }

  void _closeKeyboard() {
    print("Closing keyboard (and disconnecting from IME)");
    _keyboardController.close();
  }

Usually, before I exit the route, I will have called endEditing(), which closes the keyboard controller:

  void _endEditing() {
    print("End editing");
    _keyboardController.close();
    _composer.selection = null;

    // If we clear SuperEditor's selection, but leave SuperEditor focused, then
    // SuperEditor will automatically place the caret at the end of the document.
    // This is because SuperEditor always expects a place for text input when it
    // has focus. To prevent this from happening, we explicitly remove focus
    // from SuperEditor.
    _focusNode.unfocus();
  }

But I also tested swiping left-to-right to pop the route while the keyboard was still up (meaning endEditing() was never called), and the error still appears unless I wrap the closeKeyboard call in the delegate check.

The error in question is the assertion at line 124 of SoftwareKeyboardController:

  /// Closes the software keyboard.
  void close() {
    assert(hasDelegate);
    _delegate?.close();
  }

At this point, I'm not sure if this is indeed a bug or if one should simply default to using the delegate check.

@matthew-carroll
Copy link
Contributor Author

@Jethro87 I adjusted the SoftwareKeyboardOpener widget to detach itself at the end of the frame, instead of detaching immediately during dispose(). That should give all ancestor widgets a chance to call close() in their own dispose() method, before the controller becomes useless. Let me know if that works.

@Jethro87
Copy link
Contributor

Jethro87 commented Jan 9, 2023

@matthew-carroll That worked 👍

@matthew-carroll
Copy link
Contributor Author

@Jethro87 do you feel good about us merging this in at this point?

@Jethro87
Copy link
Contributor

Jethro87 commented Jan 9, 2023

@matthew-carroll Yes. I'm good with merging. Thanks.

@matthew-carroll
Copy link
Contributor Author

Note for the future, this PR might bring back an issue that we worked around in #791

I tried to reproduce that issue on this branch and I couldn't reproduce it. The change I made in this PR was to bring back an Exception that we recently removed. If we start hitting that exception again, we can revisit the removal of the exception.

@matthew-carroll matthew-carroll merged commit 2ef77ee into main Jan 10, 2023
@matthew-carroll matthew-carroll deleted the 875_panel-behind-keyboard branch January 10, 2023 04:07
matthew-carroll added a commit that referenced this pull request Jan 10, 2023
…st when keyboard is closed (Resolves #875) (#876)

* Includes general refactoring and packaging of Super Editor IME behavior
matthew-carroll added a commit that referenced this pull request Jan 10, 2023
…st when keyboard is closed (Resolves #875) (#876)

* Includes general refactoring and packaging of Super Editor IME behavior
dxvid-pts pushed a commit to dxvid-pts/super_editor that referenced this pull request Feb 11, 2024
…st when keyboard is closed (Resolves superlistapp#875) (superlistapp#876)

* Includes general refactoring and packaging of Super Editor IME behavior
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants