Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[webview_flutter] Soft keyboard not appearing on Q #38375

Closed
mklim opened this issue Aug 13, 2019 · 17 comments · Fixed by flutter/engine#12085
Closed

[webview_flutter] Soft keyboard not appearing on Q #38375

mklim opened this issue Aug 13, 2019 · 17 comments · Fixed by flutter/engine#12085

Comments

@mklim
Copy link
Member

@mklim mklim commented Aug 13, 2019

This affects all embedded android views, not just the WebView plugin.

See issuecomment-528149744 for the root cause.

Status update 11/20: The fix is in version v1.10.3. Currently that still isn't in stable, but has rolled to all other channels.

Status update 9/16: This has been fixed in Flutter's master channel as of 9/16/2019 via 8d23bcf. However the fix still hasn't rolled to the beta, dev, or stable channels.

@Moncader

This comment has been minimized.

Copy link

@Moncader Moncader commented Aug 22, 2019

We are seeing this issue as well on Android Q.

We aren't using webview_flutter and have a custom implementation of WebView inside a PlatformView.

However the same issue occurs.
Simply creating a WebView instance inside a PlatformView causes the following error to appear in logcat.

W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.onConnectedToRenderProcess(PG:502)
W/InputMethodManager(20606): 	at org.chromium.content.browser.framehost.NavigationControllerImpl.nativeLoadUrl(Native Method)
W/InputMethodManager(20606): 	at org.chromium.content.browser.framehost.NavigationControllerImpl.a(PG:67)
W/InputMethodManager(20606): 	at org.chromium.android_webview.AwContents.b(PG:606)
W/InputMethodManager(20606): 	at org.chromium.android_webview.AwContents.a(PG:436)
W/InputMethodManager(20606): 	at com.android.webview.chromium.WebViewChromium.loadUrl(PG:243)
W/InputMethodManager(20606): 	at android.webkit.WebView.loadUrl(WebView.java:715)
...............................

When you actually try to pull out the keyboard the following happens.

W/InputMethodManager(20606): b/117267690: Display ID mismatch found. ViewRootImpl displayId=229 InputMethodManager displayId=0. Use the right InputMethodManager instance to avoid performance overhead.
W/InputMethodManager(20606): java.lang.Throwable
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.getFallbackInputMethodManagerIfNecessary(InputMethodManager.java:537)
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.restartInput(InputMethodManager.java:1578)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.o(PG:314)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.updateState(PG:207)
W/InputMethodManager(20606): 	at android.os.MessageQueue.nativePollOnce(Native Method)
W/InputMethodManager(20606): 	at android.os.MessageQueue.next(MessageQueue.java:336)
W/InputMethodManager(20606): 	at android.os.Looper.loop(Looper.java:174)
W/InputMethodManager(20606): 	at android.app.ActivityThread.main(ActivityThread.java:7356)
W/InputMethodManager(20606): 	at java.lang.reflect.Method.invoke(Native Method)
W/InputMethodManager(20606): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
W/InputMethodManager(20606): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
W/InputMethodManager(20606): b/117267690: Display ID mismatch found. ViewRootImpl displayId=229 InputMethodManager displayId=0. Use the right InputMethodManager instance to avoid performance overhead.
W/InputMethodManager(20606): java.lang.Throwable
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.getFallbackInputMethodManagerIfNecessary(InputMethodManager.java:537)
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.showSoftInput(InputMethodManager.java:1409)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.m(PG:227)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.updateState(PG:209)
W/InputMethodManager(20606): 	at android.os.MessageQueue.nativePollOnce(Native Method)
W/InputMethodManager(20606): 	at android.os.MessageQueue.next(MessageQueue.java:336)
W/InputMethodManager(20606): 	at android.os.Looper.loop(Looper.java:174)
W/InputMethodManager(20606): 	at android.app.ActivityThread.main(ActivityThread.java:7356)
W/InputMethodManager(20606): 	at java.lang.reflect.Method.invoke(Native Method)
W/InputMethodManager(20606): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
W/InputMethodManager(20606): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)

It is noted in the Android Q release notes that WebView's don't work for some applications though there is no information on what that means. Is that the same issue and can we expect it to be fixed by Android Q's final release?

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Aug 22, 2019

Thanks for the extra logs @Moncader, that's extremely helpful. We do render the WebView in a different display from the rest of the Flutter UI, so the log talking about a "Display ID mismatch" sounds like it's likely related.

@Moncader

This comment has been minimized.

Copy link

@Moncader Moncader commented Aug 25, 2019

@Hixie @mklim
Sorry to ping you Hixie, but with the final Android 10 releases coming out, it would be great if we could:

  • Get an update on what the problem exactly is
  • If the team is looking to fix this before the official Android 10 release
  • If this is a Flutter bug or an Android 10 bug, and if the latter, if that Android bug (degradation) is going to be fixed before final.

Thanks in advanced!

@mosil

This comment has been minimized.

Copy link

@mosil mosil commented Aug 25, 2019

I has the same log like this on Anrdoid Q, too.

When you actually try to pull out the keyboard the following happens.

W/InputMethodManager(20606): b/117267690: Display ID mismatch found. ViewRootImpl displayId=229 InputMethodManager displayId=0. Use the right InputMethodManager instance to avoid performance overhead.
W/InputMethodManager(20606): java.lang.Throwable
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.getFallbackInputMethodManagerIfNecessary(InputMethodManager.java:537)
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.restartInput(InputMethodManager.java:1578)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.o(PG:314)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.updateState(PG:207)
W/InputMethodManager(20606): 	at android.os.MessageQueue.nativePollOnce(Native Method)
W/InputMethodManager(20606): 	at android.os.MessageQueue.next(MessageQueue.java:336)
W/InputMethodManager(20606): 	at android.os.Looper.loop(Looper.java:174)
W/InputMethodManager(20606): 	at android.app.ActivityThread.main(ActivityThread.java:7356)
W/InputMethodManager(20606): 	at java.lang.reflect.Method.invoke(Native Method)
W/InputMethodManager(20606): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
W/InputMethodManager(20606): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
W/InputMethodManager(20606): b/117267690: Display ID mismatch found. ViewRootImpl displayId=229 InputMethodManager displayId=0. Use the right InputMethodManager instance to avoid performance overhead.
W/InputMethodManager(20606): java.lang.Throwable
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.getFallbackInputMethodManagerIfNecessary(InputMethodManager.java:537)
W/InputMethodManager(20606): 	at android.view.inputmethod.InputMethodManager.showSoftInput(InputMethodManager.java:1409)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.m(PG:227)
W/InputMethodManager(20606): 	at org.chromium.content.browser.input.ImeAdapterImpl.updateState(PG:209)
W/InputMethodManager(20606): 	at android.os.MessageQueue.nativePollOnce(Native Method)
W/InputMethodManager(20606): 	at android.os.MessageQueue.next(MessageQueue.java:336)
W/InputMethodManager(20606): 	at android.os.Looper.loop(Looper.java:174)
W/InputMethodManager(20606): 	at android.app.ActivityThread.main(ActivityThread.java:7356)
W/InputMethodManager(20606): 	at java.lang.reflect.Method.invoke(Native Method)
W/InputMethodManager(20606): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
W/InputMethodManager(20606): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Aug 26, 2019

@Moncader we (the Flutter team) have set up a meeting with the Android team to get to the bottom of 1 and 3. This is currently a priority for me, but unfortunately I can't give a specific timeline for when this will be fixed.

@mosil thanks for the stack trace. Also please remember to 👍 the issue if you can, we use thumbs as a guide for issue priority.

@Moncader

This comment has been minimized.

Copy link

@Moncader Moncader commented Sep 2, 2019

@mklim
When are you planning on talking with the Android team?
Would you mind keeping us updated?

Our apps won't work on Android Q at this rate when it's released.

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Sep 5, 2019

Met with the Android team today.

What the problem exactly is

At a high level, this bug is happening because Flutter was working around some behavior in Android by relying on some specific behavior from InputMethodManager to enable text input in embedded platform views. The IMM behavior we were relying on for our workaround has changed after a refactor in Android Q, so the original workaround isn't functioning.

Flutter's behavior so far

The entire Flutter UI is generally drawn to a SurfaceView. Normally this makes combining native Android view widgets with Flutter widgets impossible, since the SurfaceView is one large texture and can't be composited with native Android widgets. Our PlatformView API allows native Android views to be combined with Flutter UIs. It embeds Android views “inside” a Flutter UI by placing the native view inside a virtual display, and then putting the virtual display on top of the Flutter SurfaceView. This is what WebView uses.

Here's a diagram of how it all works.

                                          (Web Widget)
+-------------------------+   +------>-----------------------+
|    Default display      |   |      |    Virtual Display    |
| +---------------------+ |   |      | +-------------------+ |
| | Window 1 (focused)  | |   |      | |Window 2 (no focus)| |
| | +-----------------+ | |   |      | | +---------------+ | |
| | |   FlutterView   | | |   |      | | |               | | |
| | +-----------+     | | |   |      | | |               | | |
| | ||          |     | | |   |      | | |    WebView    | | |
| | ||Web Widget+-------------+      | | |               | | |
| | ||          |     | | |   |      | | |               | | |
| | +-----------+     | | |   |      | | |               | | |
| | +-----------------+ | |   |      | | +---------------+ | |
| +---------------------+ |   |      | +-------------------+ |
+-------------------------+   +------>-----------------------+

So far we haven't been able to focus the window inside the VirtualDisplay. There aren't Android APIs for dynamically switching window focus between displays like this. Since the platform view is never focused, we originally were never able to bring up the keyboard in an embedded view. Android doesn't show the soft keyboard and allow for text input unless the view asking for text is focused.

We've worked around the focus issue so far by proxying the InputConnection from the virtual display back to FlutterView inside the default display, and then handing it to Android from there. That way Android considers the input to be coming from a view that has focus (FlutterView), so it shows the soft keyboard. We’ve done this by overriding FlutterView#checkInputConnectionProxy so that Android uses FlutterView as a proxy for the embedded view whenever it tries to connect to the IME, instead of trying to get an InputConnection from the view directly. We then get the InputConnection from the embedded view’s onCreateInputConnection and return that from FlutterView#onCreateInputConnection for Android to get and use.

The breakage in Q

In previous versions of Android, there was a singleton instance of InputMethodManager that was responsible for calling checkInputConnectionProxy and onCreateInputConnection across all views. So we could set up a hook in FlutterView#checkInputConnectionProxy to tell the IMM to use FlutterView for the embedded view's input connection even though they were in entirely different displays and windows.

In Q the IMM has changed to be instantiated per display instead of a singleton. This was done in order to better support window focus across multiple displays.

So our proxy workaround isn't functioning on Q. The IMM instance that calls FlutterView#checkInputConnectionProxy is a completely different instance from the one trying to actually call WebView#onCreateInputConnection, so we're never able to establish the proxy back to the focused FlutterView window. Because of this the keyboard fails to show at all because the embedded platform view's window isn't focused.

The stack trace from earlier on this issue is a warning related to the refactor, but isn't the actual root cause of a failure to show the soft input.

Next steps

I have some ideas for a few potential workarounds, unfortunately no obvious clean solutions. I'm going to test them out to see if they're viable next.

There are still a lot of unknown unknowns here. I can't guarantee anything, and things may change, but even so, this is probably going to be a matter of weeks to months, not days. Sorry.

@Moncader

This comment has been minimized.

Copy link

@Moncader Moncader commented Sep 5, 2019

@mklim
Thank you for a full description on what is going on. That helps a lot and also will allow us to work on our own solutions to the problem at hand.

However with the way Android 10 handles IMM per display now, that basically means that PlatformView for Android 10 for anything that deals with input, not just WebView is dead until this is fixed.

Android 10 is now officially released and so basically you're saying Flutter doesn't support Android 10.

While I understand that the PlatformView feature is still in beta. Lots of people use it as in the real world, we need PlatformView to deal with business-related requirements (like showing a webview only login page that is impossible to edit or move to native).

Flutter/Android 10 is all Google.
Was there no internal process while Android 10 was being developed to make sure this problem would not happen? Why did it take so long, ie, after release of Android 10 to pinpoint this problem?

It would be great if going forward the Flutter team could keep a bit more sharp of an eye on OS updates to make sure there is a stable release rolling out around the time the OS main release is out in the wild.

That aside, in Flutter 1.2.xxx we had a working WebView with keyboard implementation using reflection on the Flutter engine that allowed us to shift focus to the display window.
You can do this by fooling around with enabling and disabling flags on the Window and Views.

Since Flutter 1.7.8 didn't need that anymore we have gotten rid of it, though we'll try to add that in again for Android 10 and see if it has any positive effect and share our results if it works out.

Thank you again for sharing the roots of this problem!

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Sep 5, 2019

@Moncader no problem, glad the breakdown was helpful.

I'm sorry that Flutter is broken for you here.

Was there no internal process while Android 10 was being developed to make sure this problem would not happen? Why did it take so long, ie, after release of Android 10 to pinpoint this problem?

We don't have a process for verifying all features individually on upcoming OS versions. I think what happened here is that this particular feature slipped and wasn't tested on Q specifically. So we're unfortunately reacting here.

I've filed #39920 about fixing the broader situation that allowed this to slip.

That aside, in Flutter 1.2.xxx we had a working WebView with keyboard implementation using reflection on the Flutter engine that allowed us to shift focus to the display window.
You can do this by fooling around with enabling and disabling flags on the Window and Views.

Since Flutter 1.7.8 didn't need that anymore we have gotten rid of it, though we'll try to add that in again for Android 10 and see if it has any positive effect and share our results if it works out.

Thanks for the heads up and for offering to share any results! Appreciate it.

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Sep 9, 2019

We've successfully prototyped a fix. The one downside is that there's still noisy logs from IMM about the mismatched display ID. Currently cleaning it up into a landable patch.

@Moncader

This comment has been minimized.

Copy link

@Moncader Moncader commented Sep 13, 2019

@mklim
I know it's not a common thing to do, but sadly a new flutter stable was just released. It would be great if this could be added a hotfix soon...

Reasons being 1.9.1 added support for iOS 13 and similar iOS 13 keyboard crashing issues as well.
iOS 13 isn't even released yet and you've touted support for this new OS while not supporting an already released Android 10. This is also as viewed from a user a degradation as a simple upgrade of a users system will break their apps.

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Sep 16, 2019

The fix has rolled from the engine into Flutter's master channel at 8d23bcf. That commit still hasn't rolled into any of the other channels yet.

@Moncader unfortunately I think this is going to have to wait to roll through the regular release process. Hotfixes are just meant for emergency security issues.

@nuptdzs

This comment has been minimized.

Copy link

@nuptdzs nuptdzs commented Nov 4, 2019

This affects all embedded android views, not just the WebView plugin.

See issuecomment-528149744 for the root cause.

Status update: This has been fixed in Flutter's master channel as of 9/16/2019 via 8d23bcf. However the fix still hasn't rolled to the beta, dev, or stable channels.

Any progress? When is it expected to be released to the beta?

@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Nov 5, 2019

Any progress?

The fix commit made it into release v1.10.3 (8d23bcf, see the "tags" at the bottom). Currently stable is still on v1.9.1, but beta is on v.1.10.7 as of 10/10 (release page). Dev should also have already gotten this fix as of 9/25.

So if you're still seeing this behavior on beta, that's an unexpected bug. Would be great if you could file a new issue with flutter doctor and information about the device that you're testing on.

@nuptdzs

This comment has been minimized.

Copy link

@nuptdzs nuptdzs commented Nov 6, 2019

Any progress?

The fix commit made it into release v1.10.3 (8d23bcf, see the "tags" at the bottom). Currently stable is still on v1.9.1, but beta is on v.1.10.7 as of 10/10 (release page). Dev should also have already gotten this fix as of 9/25.

So if you're still seeing this behavior on beta, that's an unexpected bug. Would be great if you could file a new issue with flutter doctor and information about the device that you're testing on.

thanks,i change flutter channel to beta and this bug fixed~

@sebkes

This comment has been minimized.

Copy link

@sebkes sebkes commented Nov 21, 2019

Thank you for the progress. I changed the flutter channel to beta. Now the keyboard appears but it is still not possible to paste from the clipboard. After selecting characters from the entered input, the options Cut, Copy, Paste and Select All show up, but clicking one of them only made these options disappear. Tested on a Pixel 2. This is an essential feature for our application and we have to wait for the fix before we can move on. Do you already work on this problem? Can you estimate how long the problem will take to be fixed? Thank you!

Flutter doctor output:
[✓] Flutter (Channel beta, v1.10.7, on Mac OS X 10.14.6 18G95, locale en-DE)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 10.3)
[✓] Android Studio (version 3.5)
[✓] Connected device (1 available)

Console output:

W/cr_AwAutofillManager(18554): WebView autofill is disabled because WebView isn't created with activity context.
[...]
E/InputMethodManager(18554): b/117267690: Failed to get fallback IMM with expected displayId=43 actual IMM#displayId=0 view=io.flutter.plugins.webviewflutter.InputAwareWebView{c21b160 VFEDHVC.. .F...... 0,0-1080,990}
D/SmartSelectEventTracker(18554): New selection session: edit-webview (20500a657c73)
D/SmartSelectEventTracker(18554):  0: SELECTION_STARTED/, range=0,1 - smart_range=0,0 (edit-webview/)
E/InputMethodManager(18554): b/117267690: Failed to get fallback IMM with expected displayId=43 actual IMM#displayId=0 view=io.flutter.plugins.webviewflutter.InputAwareWebView{c21b160 VFEDHVC.. .F...... 0,0-1080,990}
[...]
D/SmartSelectEventTracker(18554):  1: ABANDON/, range=0,1 - smart_range=0,0 (edit-webview/)
E/InputMethodManager(18554): b/117267690: Failed to get fallback IMM with expected displayId=43 actual IMM#displayId=0 view=io.flutter.plugins.webviewflutter.InputAwareWebView{c21b160 VFEDHVC.. .F...... 0,0-1080,990}
E/InputMethodManager(18554): b/117267690: Failed to get fallback IMM with expected displayId=43 actual IMM#displayId=0 view=io.flutter.plugins.webviewflutter.InputAwareWebView{c21b160 VFEDHVC.. .F...... 0,0-1080,990}
@mklim

This comment has been minimized.

Copy link
Member Author

@mklim mklim commented Nov 21, 2019

@sebkes glad to hear this specific issue is resolved for you. You can follow progress on the copy/paste issue at #24585. The current milestone for landing a fix is December 31, but like all milestones, that's a goal and may change at any point. Unfortunately there's also a decent chance that the fix for it will need to land in the flutter engine and then roll to the framework and the other channels, which will take a decent amount of time on its own as well. Sorry that you're blocked on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.