Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On iOS, VoiceOver incorrectly activated the pop-up keyboard after touching TextField #94465

Closed
luomo-pro opened this issue Dec 1, 2021 · 18 comments · Fixed by flutter/engine#52333
Assignees
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. found in release: 2.5 Found to occur in 2.5 found in release: 2.6 Found to occur in 2.6 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-ios iOS applications specifically r: fixed Issue is closed as already fixed in a newer version team-ios Owned by iOS platform team triaged-ios Triaged by iOS platform team

Comments

@luomo-pro
Copy link

When VoiceOver's typing mode is touch typing
When a finger touches the TextField and lets go, it will activate the TextField by mistake, causing the keyboard to pop up.
If you change VoiceOver's typing mode to standard typing, you won't have this problem.
But in native typing mode, no matter what type mode you touch the TextField and let go, the popup control is not activated unless the user manually double-clicks the screen.
Note: Steps for modifying different typing modes:

  1. Open VoiceOver.
  2. Find an input box and pop up the keyboard.
  3. Use the rotor gesture (press one finger on the screen and rotate the other finger around it), then release your hand when you hear "typing mode".
  4. Swipe a finger down on the screen and you'll hear three different options: Standard type, Touch Type, and Direct Touch Type.

Since I don't speak English, I use a translator, so the description may not be completely accurate, if you have any questions, please comment!

This is my code snippet:
TextField(
decoration: InputDecoration(
labelText: "user name",
hintText: "user name or email",
prefixIcon: Icon(Icons.person)
)),

I use:
Flutter (Channel beta, 2.7.0-3.1.pre

@darshankawar darshankawar added the in triage Presently being triaged by the triage team label Dec 2, 2021
@darshankawar
Copy link
Member

@luomo-pro
I tried the scenario you mentioned on latest stable, master and beta (2.8) but wasn't able to replicate the behavior. What I tried:

  1. Ran provided code sample on ios device.
  2. Turned ON voiceover.
  3. Voiceover focus was on the textfield as shown in below screenshot:

Screenshot 2021-12-02 at 2 06 10 PM

  1. Used rotor gesture to select typing mode and then swiped down one by one to hear direct type, standard type and direct touch type, but this didn't open the keyboard.

With what you said, does following above steps, open the keyboard for you ?

stable, master, beta flutter doctor -v
[✓] Flutter (Channel stable, 2.5.3, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.5.3 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (2 days ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82


• No issues found!

[✓] Flutter (Channel master, 2.6.0-12.0.pre.883, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.6.0-12.0.pre.883 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 049ea73c4d (19 minutes ago), 2021-12-01 20:19:02 -0800
    • Engine revision c2fcadef89
    • Dart version 2.16.0 (build 2.16.0-56.0.dev)
    • DevTools version 2.8.0

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82


• No issues found!

[✓] Flutter (Channel beta, 2.8.0-3.3.pre, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.8.0-3.3.pre at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 262b70ece1 (12 hours ago), 2021-12-01 13:00:48 -0800
    • Engine revision 06a7363b0c
    • Dart version 2.15.0 (build 2.15.0-268.18.beta)

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.0.0.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 96.0.4664.55

! Doctor found issues in 1 category.





@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 2, 2021
@luomo-pro
Copy link
Author

@darshankawar
Sorry for the inaccuracy of my description.
By default, the typing mode of voiceover is standard typing.
You need to modify the typing mode to touch typing first by following the steps below:
1.Turned ON voiceover.
2. Let the focus on a textfield, and double-click to make the keyboard pop up.
3.Used rotor gesture to select
typing mode
and then swiped down one by one to hear
When you hear "touch type", please let go. At this time, you have completed the modification of the typing mode.

After you perform the above steps, in the flutter application, let go after a finger touches the textfield, you will find that the textfield is activated and the keyboard pops up directly.
You can test on the interface with multiple textfields, and you will find that when you touch which textfield and let go, which textfield will be activated.
But if you follow the above steps and change the typing mode to standard typing, there will be no such problem.
In native, no matter any typing mode, there will be no such problem.

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 2, 2021
@darshankawar
Copy link
Member

@luomo-pro
Thanks for the details and updates. I tried based on your description but observed following:

1.Turned ON voiceover.
2. Let the focus on a textfield, and double-click to make the keyboard pop up.
3.Used rotor gesture to select
typing mode
and then swiped down one by one to hear
When you hear "touch type", please let go. 

I see the keyboard still open as I popped up in step 2 above.

If I follow first 2 steps without popping the keyboard and follow rest of the steps and select touch type, it didn't pop up keyboard for me.
Please see the attached video showing the same.

RPReplay-Final1638515151.MP4

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 3, 2021
@luomo-pro
Copy link
Author

hi
@darshankawar
I'm glad I saw you successfully switched between different typing modes.
The steps I mentioned are actually just for you to switch the typing mode successfully. As for the operation step 2, it doesn’t matter whether the keyboard pops up or not.
The problem I'm talking about is the problem that occurs after changing the typing mode to touch typing.
When you change the typing mode to touch typing, please use the following steps to verify the problem I said:

  1. Reopen a flutter application.
  2. Open a page with many textfields.
    Note that the autoFocus of these textfields should all be false, so that the keyboard will not automatically pop up when this page is opened.
    If the keyboard pops up directly after opening the page, you will not be able to visually see the problem I mentioned.
  3. At this time, please touch any textfield. When you let go, you will see the keyboard pop up directly. This is a wrong behavior.
    You can also try to touch different textfields, you should see which textfield you touch, which textfield will get the input focus, that is, it is activated by mistake.
    If you change the typing mode to standard typing, and then repeat these steps, you will find that after touching the textfield, it will not get the input focus, will not be activated, and the keyboard will not pop up.
    Only if you touch the textfield and double-tap the screen, the textfield will be activated, the input focus will be gained, and the keyboard will pop up.

In native, even if the typing mode is touch typing, it is the same as standard typing. You must double-click the screen to get the textfield to get the input focus.

Do you understand what I said above?
If the problem cannot be reproduced, I can record a video, just because I am using Chinese, I don’t know if you can understand it.

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 3, 2021
@darshankawar
Copy link
Member

Thanks for the details which helped to replicate the issue properly using below code sample.

import 'package:flutter/material.dart';

void main() {
  runApp(MaterialApp(
    home: Scaffold(
      body: Center(
        child: TextField(
          decoration: InputDecoration(
            labelText: 'name',
            hintText: 'name',
            prefixIcon: Icon(Icons.person)
          ),
        )
      )
    )
  ));
}

stable, master flutter doctor -v
[✓] Flutter (Channel stable, 2.5.3, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.5.3 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (2 days ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82


• No issues found!

[✓] Flutter (Channel master, 2.6.0-12.0.pre.883, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.6.0-12.0.pre.883 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 049ea73c4d (19 minutes ago), 2021-12-01 20:19:02 -0800
    • Engine revision c2fcadef89
    • Dart version 2.16.0 (build 2.16.0-56.0.dev)
    • DevTools version 2.8.0

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 93.0.4577.82


• No issues found!

@darshankawar darshankawar added a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) platform-ios iOS applications specifically found in release: 2.5 Found to occur in 2.5 found in release: 2.6 Found to occur in 2.6 has reproducible steps The issue has been confirmed reproducible and is ready to work on a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. framework flutter/packages/flutter repository. See also f: labels. and removed in triage Presently being triaged by the triage team labels Dec 3, 2021
@chinmaygarde chinmaygarde added the P2 Important issues not at the top of the work list label Dec 6, 2021
@wapscccs
Copy link

wapscccs commented Dec 20, 2021

I am also a blind developer. I think blind friends who use accessibility will encounter this problem. This problem has caused them serious troubles. I hope it can be resolved as soon as possible. Thank you very much to the Flutter team.
@darshankawar @chinmaygarde @justinmc @LongCatIsLooong

@lilangtop
Copy link

I'm writing a software for office work that uses a lot of TextField, and I've had a lot of feedback from blind users about this issue.
Many blind people use touch typing mode, which causes them to touch the TextField very carefully and activate it by mistake if they are not careful, which is a terrible experience.
This issue has been around for a long time, any progress?

cc
@justinmc
@LongCatIsLooong
@chunhtai

@chunhtai
Copy link
Contributor

it sounds like iOS is sending the touch event differently when the touch typing is enabled, we may need to do a special treament if that setting is turned on

@daiweiqiang6688
Copy link

I have the same problem, after touching textfield, it is activated by mistake and the keyboard pops up automatically.
Is there any progress or solution now?

@c469591
Copy link

c469591 commented Mar 31, 2022

I also met the same problem, TextField will automatically get focus and pop-up keyboard, this problem has existed for a long time, what is the progress now?
@chunhtai

@Nuno69
Copy link

Nuno69 commented Apr 19, 2022

Hello,
here are my findings. In the meantime, I am working to find a decently easy to implement workarround.
The problem seems not too complicated or diverse. What seems to happen is that when the finger is lifted up from the text field, VoiceOver seems to execute its default Activate action, the same one that is executed on double tap. This might lead to a conclusion that something in the internal implementation of the FLutter's text box widget is making it active even before the keyboard is popped up, and so VoiceOver reacts to it as it were a software keyboard key.
Now I am inspecting the user interface structure and the implementation of the FLutter's Text Box to hopefully give you more insight on that.

@flutter-triage-bot flutter-triage-bot bot added multiteam-retriage-candidate team-ios Owned by iOS platform team triaged-ios Triaged by iOS platform team labels Jul 8, 2023
@s17476
Copy link

s17476 commented Dec 2, 2023

Hey guys. I see that the last post is from a year and a half ago, but the problem still persists in the current version of Flutter. I managed to get around it very easily. Here: https://github.com/s17476/semantic_text_field  I made a repo presenting the operation and below is my solution. Of course, this is the minimum code sample needed for TextField to function properly.

class SemanticTextField extends StatefulWidget {
  const SemanticTextField({
    Key? key,
    this.decoration,
    this.controller,
    this.onChanged,
  }) : super(key: key);

  final InputDecoration? decoration;
  final TextEditingController? controller;
  final Function(String)? onChanged;

  @override
  State<SemanticTextField> createState() => _SemanticTextFieldState();
}

class _SemanticTextFieldState extends State<SemanticTextField> {
  late final TextEditingController _textEditingController;
  final FocusNode _focusNode = FocusNode();
  bool _isKeyboardVisible = false;

  String? _getSemanticLabel() {
    if (Platform.isAndroid) {
      return null;
    }

    // Add translations here

    // Textfield
    final textFieldLabel = 'Pole tekstowe';
    // Edition
    final editingLabel = 'Edycja';
    // Double tap to edit
    final doubleTapLabel = 'Stuknij dwukrotnie aby edytować';

    var label = _textEditingController.text.isEmpty
        ? widget.decoration?.hintText ?? ''
        : '';
    label += ';$textFieldLabel';
    label += ';${_isKeyboardVisible ? editingLabel : doubleTapLabel}';

    return label;
  }

  @override
  void initState() {
    _textEditingController = widget.controller ?? TextEditingController();
    _focusNode.addListener(() {
      setState(() {
        _isKeyboardVisible = _focusNode.hasFocus;
      });
    });
    super.initState();
  }

  @override
  void dispose() {
    if (widget.controller == null) {
      _textEditingController.dispose();
    }
    _focusNode.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Semantics(
      value: Platform.isIOS ? _textEditingController.text : null,
      label: _getSemanticLabel(),
      child: ExcludeSemantics(
        excluding: Platform.isIOS,
        child: TextField(
          controller: _textEditingController,
          focusNode: _focusNode,
          decoration: widget.decoration ?? const InputDecoration(),
          onChanged: (value) {
            setState(() {});
            widget.onChanged?.call(value);
          },
        ),
      ),
    );
  }
}

I hope this will be useful to someone. Our blind consultant is satisfied with the operation of this solution.

@luomo-pro
Copy link
Author

@s17476
Hi, I tried your solution and indeed the keyboard does not pop up automatically.
But now blind people can't move the character cursor using the voice over and can't edit text well.
This problem I feel may be worse than the keyboard pop up automatically.
Anyway, thanks for the solution, thanks!

@s17476
Copy link

s17476 commented Dec 4, 2023

@luomo-pro
I know that when the TextField gains focus, you cannot jump between words. You need to enter a character first. Then it works as it should. Or at least that's what I was told. It is quite possible that my solution could be further improved. Maybe we need to play around with unlocking the semantics after the field gets focus. I haven't tried it.

@justinmc
Copy link
Contributor

justinmc commented Apr 1, 2024

I was able to reproduce this with darshankawar's code with the "Touch Typing" setting in the Voice Over settings under "Typing". There is another option "Direct Touch Typing", but that one doesn't repro the problem.

When using "Touch Typing", the framework is receiving a tap event (here) when the text field is tapped once, but in any other typing mode, no tap event is received.

@jmagman Do you have any insight into how VoiceOver affects how Flutter receives touch events? If not then @chunhtai probably does, though he's on leave until next week.

@jmagman
Copy link
Member

jmagman commented Apr 4, 2024

@jmagman Do you have any insight into how VoiceOver affects how Flutter receives touch events?

Hm, I've never looked into how that works. If @chunhtai doesn't have the bandwidth to dig into this soon, I'll see if I can get someone to debug this, particularly since it's reproducible (cc @vashworth who might be the person who gets pulled into this 🙂 )

@chunhtai
Copy link
Contributor

chunhtai commented Apr 8, 2024

Hi @jmagman
I currently don't have the bandwidth to look into it. Per my previous comment, It seems like voiceover is sending pointer touch directly when touch typing is on, and that activate the textfield and pop up keyboard. I have not looked into how to fix it yet.

@vashworth
Copy link
Contributor

vashworth commented Apr 22, 2024

Not sure on solution, but here's what I think is happening:

Seems like when touch typing is on, accessibilityActivate is triggered after only one tap. Whereas with standard typing, it's triggered after double tapping.

https://github.com/flutter/engine/blob/33c828fb3ff5c7628dbb01fbf40ed95831ad00d7/shell/platform/darwin/ios/framework/Source/accessibility_text_entry.mm#L262-L267

I believe this is happening due to Flutter adding the UIAccessibilityTraitKeyboardKey accessibility trait to TextInputSemanticsObject.

https://github.com/flutter/engine/blob/33c828fb3ff5c7628dbb01fbf40ed95831ad00d7/shell/platform/darwin/ios/framework/Source/accessibility_text_entry.mm#L307-L312

According to Apple's documentation UIAccessibilityTraitKeyboardKey tells iOS that the "accessibility element behaves like a keyboard key". And if we look at the different typing modes, Touch typing reacts on a single tap whereas Standard typing reacts on a double tap:

Standard typing: Select a key by swiping left or right on the keyboard, then double-tap to enter the character.
Touch typing: Touch a key on the keyboard to select it, then lift your finger to enter the character.

So by treating TextInputSemanticsObject as a keyboard key, it reacts to a single tap when touch typing is enabled.


Update: I confirmed that UIAccessibilityTraitKeyboardKey is what's causing this behavior.

@vashworth vashworth self-assigned this Apr 23, 2024
auto-submit bot pushed a commit to flutter/engine that referenced this issue Apr 24, 2024
`UIAccessibilityTraitKeyboardKey` was added in #4575 so that `TextInputSemanticsObject` supported VoiceOver gestures for text editing features, such as pinch to select text, and up/down fling to move cursor. After experimenting with it, I found that those features were still available even after removing `UIAccessibilityTraitKeyboardKey`. 

Fixes flutter/flutter#94465.

In Touch Typing Mode:

https://github.com/flutter/engine/assets/15619084/ccfe90ff-d3bc-427b-b1aa-9ec1242c0c89

In Standard Typing Mode:

https://github.com/flutter/engine/assets/15619084/c78b1fb0-0816-41fb-9dd5-ed8b8a4cda2d

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
@darshankawar darshankawar added the r: fixed Issue is closed as already fixed in a newer version label Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. found in release: 2.5 Found to occur in 2.5 found in release: 2.6 Found to occur in 2.6 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P2 Important issues not at the top of the work list platform-ios iOS applications specifically r: fixed Issue is closed as already fixed in a newer version team-ios Owned by iOS platform team triaged-ios Triaged by iOS platform team
Projects
None yet
Development

Successfully merging a pull request may close this issue.