Native ios context menu

[justinmc]

In order to work around the fact that iOS 15 shows a notification on pressing Flutter's paste button (#103163), this PR allows showing the iOS system context menu in text fields.

It is currently opt-in, which a user would typically do like this (also in example system_context_menu.0.dart):

      contextMenuBuilder: (BuildContext context, EditableTextState editableTextState) {
        // If supported, show the system context menu.
        if (SystemContextMenu.isSupported(context)) {
          return SystemContextMenu.editableText(
            editableTextState: editableTextState,
          );
        }
        // Otherwise, show the flutter-rendered context menu for the current
        // platform.
        return AdaptiveTextSelectionToolbar.editableText(
          editableTextState: editableTextState,
        );
      },

Requires engine PR flutter/engine#50095.

API changes

SystemContextMenu

A widget that shows the system context menu when built, and removes it when disposed. The main high-level way that most users would use this PR. Only works on later versions of iOS.

SystemContextMenuController

Used under the hood to hide and show a system context menu. There can only be one visible at a time.

MediaQuery.supportsShowingSystemContextMenu

Sent by the iOS embedder to tell the framework whether or not the platform supports showing the system context menu. That way the framework, or Flutter developers, can decide to show a different menu.

flutter/platform ContextMenu.showSystemContextMenu

Sent by the framework to show the menu at a given targetRect, which is the current selection rect.

flutter/platform ContextMenu.hideSystemContextMenu

Sent by the framework to hide the menu. Typically not needed, because the platform will hide the menu when the user taps outside of it and after the user presses a button, but it handles edge cases where the user programmatically rebuilds the context menu, for example.

flutter/platform System.onDismissSystemContextMenu

Sent by the iOS embedder to indicate that the system context menu has been hidden by the system, such as when the user taps outside of the menu. This is useful when there are multiple instances of SystemContextMenu, such as with multiple text fields.

[hellohuanlin]

also check if it's for text field vs non-text field widget?

[cbracken]

Reason for revert: unresolved docs links. See failure here: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20docs_test/16540/overview

dartdoc:stdout: Generating docs for package flutter...
dartdoc:stderr:   error: unresolved doc reference [TextInput.showSystemContextMenu]
dartdoc:stderr:     from widgets.MediaQueryData.supportsShowingSystemContextMenu: (file:///b/s/w/ir/x/w/flutter/packages/flutter/lib/src/widgets/media_query.dart:579:14)
dartdoc:stderr:     in documentation inherited from widgets.MediaQueryData.supportsShowingSystemContextMenu: (file:///b/s/w/ir/x/w/flutter/packages/flutter/lib/src/widgets/media_query.dart:579:14)
dartdoc:stderr:   error: unresolved doc reference [showSystemContextMenu]
dartdoc:stderr:     from services.SystemContextMenuController.hide: (file:///b/s/w/ir/x/w/flutter/packages/flutter/lib/src/services/text_input.dart:2554:16)
dartdoc:stderr:   error: unresolved doc reference [hideSystemContextMenu]
dartdoc:stderr:     from services.SystemContextMenuController.show: (file:///b/s/w/ir/x/w/flutter/packages/flutter/lib/src/services/text_input.dart:2509:16)

[justinmc]

Whoops, how did that not get caught in presubmit? Well I'll create a new PR and fix those.

[matthew-carroll]

@justinmc is there a reason that the new widget for displaying the toolbar only works with Flutter's built-in text field implementation? It has a private constructor and the factory constructor requires a EditableTextState.

[matthew-carroll]

@justinmc I'm also curious, is a Rect truly the fundamental required geometry? Is that what iOS demands? Or does it really want an Offset and Rect was used in the API because that happens to fit better with Flutter's existing text field implementation?

I ask that because in SuperTextField, while we expose the text layout, most of our toolbar work is in terms of a predetermined focal point.

[matthew-carroll]

@justinmc I'm continuing integration with this and I have another question. This widget, and surrounding classes, are all given platform-agnostic names. For example: SystemContextMenu instead of IOSSystemContextMenu.

Do you have meaningful evidence that a singular API will be able to work across all platforms? As far as I can tell this work has only attempted to integrate with iOS APIs. I'm wondering if there's a good chance that over time this API (especially once you add the ability to customize the buttons) will only work for iOS due to platform API differences. If so, at that point Flutter will be stuck with an iOS-specific class called SystemContextMenu, which will then be joined by an AndroidSystemContextMenu and Flutter will forever have to explain that SystemContextMenu is only for iOS because reasons. What are your thoughts on that?

[hellohuanlin]

@matthew-carroll

I'm also curious, is a Rect truly the fundamental required geometry? Is that what iOS demands?

Assuming you are talking about the targetRect which is the rect of the selected text. Yes it's what iOS demands. iOS make use of this rect to determine where to show the menu.

Or does it really want an Offset and Rect was used in the API because that happens to fit better with Flutter's existing text field implementation?

There's an alternative iOS API that takes the offset and the arrow direction, but it won't work because there's no way to tell the size of the menu (the size can vary based on accessibility settings). So we can't tell the offset (should it be
the top or bottom of text field?) or arrow direction (pointing up or down?).

Do you have meaningful evidence that a singular API will be able to work across all platforms? As far as I can tell this work has only attempted to integrate with iOS APIs. I'm wondering if there's a good chance that over time this API (especially once you add the ability to customize the buttons) will only work for iOS due to platform API differences.

Currently we don't see a need to implement this on Android. However, we did try to make the API platform agnostic, if such need arise in the future.

The current milestone requires developers to adopt this API manually, since it lacks of feature parity (e.g. look up, search web, custom actions). In the next milestone we want to enable it by default, so that developers get system menu on iOS 16+, and Flutter menu on other platforms.

[matthew-carroll]

Currently we don't see a need to implement this on Android. However, we did try to make the API platform agnostic, if such need arise in the future.

Ok, as I mentioned in my previous comment, I would expect that this is more likely to end up in a bad place than a good place. If you and @justinmc haven't inspected these APIs on Android/Windows/Linux, then there's no indication that the API you've created here would work at all with any other platform. I think if Flutter is going to bring in little pieces of specific platforms, it's much safer to name them after that platform, which is what it really represents anyway. Then, after all platforms are accumulated, you can create a lowest common denominator on top, if desired.

[hellohuanlin]

@matthew-carroll Note that once milestone 2 is done, developers should get secure paste out of box. I am not expecting them to use these types directly. I believe they are only exposed so that we can share code between Cupertino and Material.

I am not an expert on flutter/dart tho, so I will ask for @justinmc's opinion.

[matthew-carroll]

I am not expecting them to use these types directly.

Please do NOT make that assumption. This is a long-lived blind spot for Flutter framework developers. @justinmc I'd really appreciate it if you would socialize this problem. Assume that EVERYTHING will be used externally. The reason I'm commenting on this is precisely because I'm adding the use of these APIs to SuperTextField and SuperEditor, neither of which uses any of Flutter's built-in RenderEditable system. Cupertino and Material do not define the universe. They're just two arbitrary UI themes.

[justinmc]

EditableTextState requirement

The reason why the constructor requires EditableTextState is because currently the system context menu won't work without an active text input connection, so I'm trying to lock it down for the time being to avoid people using it in situations where the native menu won't work. I'd like to generalize it in the future so that it can work with arbitrary buttons that may not have to do with any text input connection.

Maybe instead of taking an EditableTextState it should take an interface that EditableTextState implements? What do you think?

SystemContextMenu naming

I think I'm probably in agreement about the SystemContextMenu name now that I'm facing a similar situation in trying to support Android's Scribe and iOS's Scribble... I think I'm going to try to expose the native APIs to the framework more or less verbatim, and then build any abstraction on top of that. I'll see how it goes over there and then see if anything can be done for SystemContextMenu going forward.

@matthew-carroll you've probably seen SystemContextMenuController, will that work for your case? There is an assert about needing a text input connection, but it's meant to be more generic.

[matthew-carroll]

I think I'm going to try to expose the native APIs to the framework more or less verbatim, and then build any abstraction on top of that.

I'm glad you've finally joined me in that view! 😅

There is an assert about needing a text input connection, but it's meant to be more generic.

That's probably fine for us. Though, if iOS technically makes it possible to show and orient a toolbar without an active IME connection, then I don't know why Flutter would want to add that restriction.

[hellohuanlin]

I'm glad you've finally joined me in that view! 😅

I am happy with this too, as long as both of you agree on a long term vision. I suspect Android may introduce a similar restriction and we should have a good plan for that. Ideally most developers shouldn't have to deal with things like which platform they are working on, and menu's arrow direction.

That's probably fine for us. Though, if iOS technically makes it possible to show and orient a toolbar without an active IME connection, then I don't know why Flutter would want to add that restriction.

It is non-trivial effort to support showing iOS edit menu for non-text field in the engine. We would need a UIView similar to FlutterTextInput to be served as a UIEditMenuInteractionDelegate.

[justinmc]

That's probably fine for us. Though, if iOS technically makes it possible to show and orient a toolbar without an active IME connection, then I don't know why Flutter would want to add that restriction.

It is non-trivial effort to support showing iOS edit menu for non-text field in the engine. We would need a UIView similar to FlutterTextInput to be served as a UIEditMenuInteractionDelegate.

+1, if/when we add support for that we should remove that assert.