Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect text selection toolbar position in scaled Platform Views (also WebView) at least Flutter 3.13.9 on Android #144685

Open
toda-bps opened this issue Mar 6, 2024 · 4 comments
Labels
a: platform-views Embedding Android/iOS views in Flutter apps found in release: 3.19 Found to occur in 3.19 found in release: 3.21 Found to occur in 3.21 has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-android Android applications specifically team-android Owned by Android platform team triaged-android Triaged by Android platform team

Comments

@toda-bps
Copy link

toda-bps commented Mar 6, 2024

Steps to reproduce

Related: #144278

  1. Run code sample.
  2. LongPress any text on the page.

Expected results

Text selection toolbar and selection range handles are in the correct place.

Actual results

Text selection toolbar and selection range handles are in the wrong place (see video).

Code sample

Code sample
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';

void main() => runApp(const _App());

class _App extends StatelessWidget {
  const _App();

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(home: _Home());
  }
}

class _Home extends StatefulWidget {
  const _Home();

  @override
  State<_Home> createState() => _HomeState();
}

class _HomeState extends State<_Home> {
  final _controller = WebViewController();

  @override
  void initState() {
    super.initState();
    _controller
      ..setJavaScriptMode(JavaScriptMode.unrestricted)
      ..loadRequest(Uri.https('api.flutter.dev'));
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Scaled WebView Tester'),
      ),
      body: Transform.scale(
        scale: 0.75,
        child: WebViewWidget(controller: _controller),
      ),
    );
  }
}

Screenshots or Video

Video demonstration
3.13.9_text-selection-toolbar.mp4

Logs

No response

Flutter Doctor output

Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel [user-branch], 3.13.9, on macOS 14.3.1 23D60 darwin-x64, locale ja-JP)
    ! Flutter version 3.13.9 on channel [user-branch] at /System/Volumes/Data/buildroot/flutter_3.13
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at https://flutter.dev/docs/get-started/install.
    ! Upstream repository unknown source is not a standard remote.
      Set environment variable "FLUTTER_GIT_URL" to unknown source to dismiss this error.
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.1)
[✓] VS Code (version 1.86.2)
[✓] Connected device (2 available)
[✓] Network resources
@moffatman
Copy link
Contributor

There are various problems with scale transformations interaction with Overlays such as text selection, I have been too lazy to file issue for each and create PR, mainly don't feel like writing tests, but you can see my fixes here

moffatman@c992cd4

@darshankawar darshankawar added the in triage Presently being triaged by the triage team label Mar 7, 2024
@darshankawar
Copy link
Member

On latest master and stable, it gives below results:

144685.mov
stable, master flutter doctor -v
[!] Flutter (Channel stable, 3.19.2, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.19.2 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 7482962148 (5 days ago), 2024-02-27 16:51:22 -0500
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.21.0-1.0.pre.47, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.21.0-1.0.pre.47 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f677027655 (87 minutes ago), 2024-03-06 22:57:32 -0500
    • Engine revision 6c1751bd77
    • Dart version 3.4.0 (build 3.4.0-207.0.dev)
    • DevTools version 2.33.1
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dhs/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13C100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 65.1.4
    • Dart plugin version 213.7228

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.29.0

[✓] Connected device (3 available)
    • Darshan's iphone (mobile) • 21150b119064aecc249dfcfe05e259197461ce23 • ios
      • iOS 15.3.1 19D52
    • macOS (desktop)           • macos                                    •
      darwin-x64     • macOS 12.2.1 21D62 darwin-x64
    • Chrome (web)              • chrome                                   •
      web-javascript • Google Chrome 109.0.5414.119

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.
      
[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.



@darshankawar darshankawar added platform-android Android applications specifically a: platform-views Embedding Android/iOS views in Flutter apps has reproducible steps The issue has been confirmed reproducible and is ready to work on found in release: 3.19 Found to occur in 3.19 found in release: 3.21 Found to occur in 3.21 team-android Owned by Android platform team and removed in triage Presently being triaged by the triage team labels Mar 7, 2024
@reidbaker reidbaker added P2 Important issues not at the top of the work list triaged-android Triaged by Android platform team labels Mar 28, 2024
@justinmc
Copy link
Contributor

@moffatman your fixes look about right, I'll go ahead and try to open a PR for those if you don't mind.

Unassigning @johnmccutchan and assigning myself.

@justinmc justinmc assigned justinmc and unassigned johnmccutchan Mar 29, 2024
justinmc added a commit to justinmc/flutter that referenced this issue Mar 29, 2024
@justinmc justinmc removed their assignment Apr 2, 2024
@justinmc
Copy link
Contributor

justinmc commented Apr 2, 2024

Unassigning myself as it looks like this needs more work at sniffing out all the places where transforms are not applied properly. My PR will still move ahead but it won't fix this issue completely. The PR includes most of @moffatman's fixes as well as a few of my own that I noticed while trying out a TextField inside of a Transform.scale (which should now work). However, putting a transform above MaterialApp still doesn't work with my PR.

auto-submit bot pushed a commit that referenced this issue Apr 8, 2024
Fixes bugs in the text selection positioning calculations so that they work even when the field is scaled. In many cases, we were simply translating things around without applying the proper localToGlobal (or vice versa) transform.

| Before | After |
| --- | --- |
| <img src="https://github.com/flutter/flutter/assets/389558/a5a45472-98c5-4cdf-b382-218971fd9404" /> | <img src="https://github.com/flutter/flutter/assets/389558/f396a1af-2546-4e38-a9d9-6c6edfa38d94" /> |

Partial fix for: #144685
It looks like there are other problems where transforms aren't applied properly. Putting a transform at the root of the application, above MaterialApp, will expose more problems.

<details>

<summary>Sample code</summary>

```dart
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';

void main() => runApp(const _App());

class _App extends StatelessWidget {
  const _App();

  @OverRide
  Widget build(BuildContext context) {
    return const MaterialApp(home: _Home());
  }
}

class _Home extends StatefulWidget {
  const _Home();

  @OverRide
  State<_Home> createState() => _HomeState();
}

class _HomeState extends State<_Home> {
  final _controller = WebViewController();
  final TextEditingController textEditingController = TextEditingController(
    text: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.',
  );
  final OverlayPortalController overlayPortalController = OverlayPortalController();

  @OverRide
  void initState() {
    super.initState();
    _controller
      ..setJavaScriptMode(JavaScriptMode.unrestricted)
      ..loadRequest(Uri.https('api.flutter.dev'));
  }

  @OverRide
  Widget build(BuildContext context) {
    overlayPortalController.show();
    return Scaffold(
      appBar: AppBar(
        title: const Text('Scaled WebView Tester'),
      ),
      body: Stack(
        children: <Widget>[
          Transform.scale(
            alignment: Alignment.topLeft,
            scale: 0.5,
            child: TextField(
              controller: textEditingController,
              maxLines: null,
            ),
          ),
          OverlayPortal(
            controller: overlayPortalController,
            overlayChildBuilder: (BuildContext context) {
              return Positioned(
                top: 0.0,
                left: 0.0,
                child: SizedBox(
                  height: 1000,
                  width: 1000,
                  child: Stack(
                    children: <Widget>[
                      Positioned(
                        top: 90.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                      Positioned(
                        top: 102.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                    ],
                  ),
                ),
              );
            },
          ),
        ],
      ),
    );
  }
}
```

</details>
TahaTesser pushed a commit to TahaTesser/flutter that referenced this issue Apr 9, 2024
Fixes bugs in the text selection positioning calculations so that they work even when the field is scaled. In many cases, we were simply translating things around without applying the proper localToGlobal (or vice versa) transform.

| Before | After |
| --- | --- |
| <img src="https://github.com/flutter/flutter/assets/389558/a5a45472-98c5-4cdf-b382-218971fd9404" /> | <img src="https://github.com/flutter/flutter/assets/389558/f396a1af-2546-4e38-a9d9-6c6edfa38d94" /> |

Partial fix for: flutter#144685
It looks like there are other problems where transforms aren't applied properly. Putting a transform at the root of the application, above MaterialApp, will expose more problems.

<details>

<summary>Sample code</summary>

```dart
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';

void main() => runApp(const _App());

class _App extends StatelessWidget {
  const _App();

  @OverRide
  Widget build(BuildContext context) {
    return const MaterialApp(home: _Home());
  }
}

class _Home extends StatefulWidget {
  const _Home();

  @OverRide
  State<_Home> createState() => _HomeState();
}

class _HomeState extends State<_Home> {
  final _controller = WebViewController();
  final TextEditingController textEditingController = TextEditingController(
    text: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.',
  );
  final OverlayPortalController overlayPortalController = OverlayPortalController();

  @OverRide
  void initState() {
    super.initState();
    _controller
      ..setJavaScriptMode(JavaScriptMode.unrestricted)
      ..loadRequest(Uri.https('api.flutter.dev'));
  }

  @OverRide
  Widget build(BuildContext context) {
    overlayPortalController.show();
    return Scaffold(
      appBar: AppBar(
        title: const Text('Scaled WebView Tester'),
      ),
      body: Stack(
        children: <Widget>[
          Transform.scale(
            alignment: Alignment.topLeft,
            scale: 0.5,
            child: TextField(
              controller: textEditingController,
              maxLines: null,
            ),
          ),
          OverlayPortal(
            controller: overlayPortalController,
            overlayChildBuilder: (BuildContext context) {
              return Positioned(
                top: 0.0,
                left: 0.0,
                child: SizedBox(
                  height: 1000,
                  width: 1000,
                  child: Stack(
                    children: <Widget>[
                      Positioned(
                        top: 90.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                      Positioned(
                        top: 102.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                    ],
                  ),
                ),
              );
            },
          ),
        ],
      ),
    );
  }
}
```

</details>
gilnobrega pushed a commit to gilnobrega/flutter that referenced this issue Apr 22, 2024
Fixes bugs in the text selection positioning calculations so that they work even when the field is scaled. In many cases, we were simply translating things around without applying the proper localToGlobal (or vice versa) transform.

| Before | After |
| --- | --- |
| <img src="https://github.com/flutter/flutter/assets/389558/a5a45472-98c5-4cdf-b382-218971fd9404" /> | <img src="https://github.com/flutter/flutter/assets/389558/f396a1af-2546-4e38-a9d9-6c6edfa38d94" /> |

Partial fix for: flutter#144685
It looks like there are other problems where transforms aren't applied properly. Putting a transform at the root of the application, above MaterialApp, will expose more problems.

<details>

<summary>Sample code</summary>

```dart
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';

void main() => runApp(const _App());

class _App extends StatelessWidget {
  const _App();

  @OverRide
  Widget build(BuildContext context) {
    return const MaterialApp(home: _Home());
  }
}

class _Home extends StatefulWidget {
  const _Home();

  @OverRide
  State<_Home> createState() => _HomeState();
}

class _HomeState extends State<_Home> {
  final _controller = WebViewController();
  final TextEditingController textEditingController = TextEditingController(
    text: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.',
  );
  final OverlayPortalController overlayPortalController = OverlayPortalController();

  @OverRide
  void initState() {
    super.initState();
    _controller
      ..setJavaScriptMode(JavaScriptMode.unrestricted)
      ..loadRequest(Uri.https('api.flutter.dev'));
  }

  @OverRide
  Widget build(BuildContext context) {
    overlayPortalController.show();
    return Scaffold(
      appBar: AppBar(
        title: const Text('Scaled WebView Tester'),
      ),
      body: Stack(
        children: <Widget>[
          Transform.scale(
            alignment: Alignment.topLeft,
            scale: 0.5,
            child: TextField(
              controller: textEditingController,
              maxLines: null,
            ),
          ),
          OverlayPortal(
            controller: overlayPortalController,
            overlayChildBuilder: (BuildContext context) {
              return Positioned(
                top: 0.0,
                left: 0.0,
                child: SizedBox(
                  height: 1000,
                  width: 1000,
                  child: Stack(
                    children: <Widget>[
                      Positioned(
                        top: 90.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                      Positioned(
                        top: 102.0,
                        left: 0.0,
                        child: Container(
                          height: 1.0,
                          width: 1000,
                          color: Colors.blue,
                        ),
                      ),
                    ],
                  ),
                ),
              );
            },
          ),
        ],
      ),
    );
  }
}
```

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: platform-views Embedding Android/iOS views in Flutter apps found in release: 3.19 Found to occur in 3.19 found in release: 3.21 Found to occur in 3.21 has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-android Android applications specifically team-android Owned by Android platform team triaged-android Triaged by Android platform team
Projects
None yet
Development

No branches or pull requests

6 participants