Skip to content

Commit

Permalink
Adapt iOS16+ dictation (#37188)
Browse files Browse the repository at this point in the history
Summary:
#19687
https://developer.apple.com/forums/thread/711413

When system version is lower than `iOS 16`, it does not support `dictation` and `keyboard` working at the same time, so if we modify the text, the system will immediately interrupt the `dictation`, so we need to prohibit modification of the text during `recording` and `recognition`

When system version is higher than `iOS 16`, `dictation` and `keyboard` can work at the same time, so `textInputMode.primaryLanguage` is no longer changed to `dictation`, so we can modify the text during `recording`, because the system will not interrupt, but we cannot modify the text during `recognition`, Because the system will temporarily add a `_UITextPlaceholderAttachment` to display the recognition `UIActivityIndicator`

## Changelog:

<!-- Help reviewers and the release process by writing your own changelog entry.

Pick one each for the category and type tags:

[ANDROID|GENERAL|IOS|INTERNAL] [BREAKING|ADDED|CHANGED|DEPRECATED|REMOVED|FIXED|SECURITY] - Message

For more details, see:
https://reactnative.dev/contributing/changelogs-in-pull-requests
-->

[IOS][FIXED] - Adapt iOS16+ dictation judge condition

Pull Request resolved: #37188

Test Plan:
Test Code
```javascript
constructor(props) {
  super(props)
  this.state = {
    value: '',
    logList: [],
  }
}

render() {
  return (
    <View style={{ flex: 1, padding: 50, backgroundColor: 'white'}}>
      <TextInput
        style={{ marginTop: 20, height: 50, borderWidth: 1, borderColor: 'black' }}
        placeholder={'please input'}
        placeholderTextColor={'gray'}
        value={this.state.value}
        onChangeText={value => {
          let logList = this.state.logList
          logList.push(value.length <= 0 ? 'null' : value.replace(/\uFFFC/g, '_uFFFC'))
          this.setState({ value, logList })
        }}
        onEndEditing={() => this.setState({ value: '', logList: [] })}
      />
      <FlatList
        style={{ marginTop: 20, maxHeight: 300, borderWidth: 1, borderColor: 'black' }}
        data={this.state.logList}
        renderItem={({item}) => (
            <Text>{item}</Text>
        )}
      />
    </View>
  )
}
```

Case A:  Required < iOS16
1. ensure that #18890 can work well and dictation will not be interrupted immediately

https://github.com/facebook/react-native/assets/20135674/e69a609c-2dc4-48fc-8186-f9e5af3ac879

Case B: Required >= iOS16
1. ensure that #18890 can work well and dictation will not be interrupted immediately

https://github.com/facebook/react-native/assets/20135674/caa97e18-c7c4-4a08-9872-b50130f73bf4

Case C: Required >= iOS16
1. start dictation
3. then do not speak any words
4. then end dictation
5. verify that `onChangeText` will callback "\uFFFC" once
6. and then verify `onChangeText` callback an empty string "" once

https://user-images.githubusercontent.com/20135674/235960378-90155ec5-a129-47bc-825b-ee6cb03e7286.MP4

Case D: Required >= iOS16
1. start dictation
3. input some text while speaking some words
4. then end dictation
5. and verify that the `onChangeText` callback work fine.

https://user-images.githubusercontent.com/20135674/235960411-e479d9ab-856a-4407-a644-986426825133.MP4

Case E: Required >= iOS16
1. start dictation
2. say a word
3. and then switch the keyboard to other language
4. verify that dictation will not end
6. continue say some word
8. verify the `onChangeText` callback work fine.

https://user-images.githubusercontent.com/20135674/235960450-351f1aaf-80c0-4d1c-b5c9-3e2cd7225875.MP4

Reviewed By: sammy-SC

Differential Revision: D45563187

Pulled By: dmytrorykun

fbshipit-source-id: 7467b313769896140434f60dcb3590d0b3c1aa15
  • Loading branch information
hellohublot authored and facebook-github-bot committed May 23, 2023
1 parent 98789e9 commit e8b4bb0
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ NS_ASSUME_NONNULL_BEGIN

@property (nonatomic, assign) BOOL contextMenuHidden;
@property (nonatomic, assign, readonly) BOOL textWasPasted;
@property (nonatomic, assign, readonly) BOOL dictationRecognizing;
@property (nonatomic, copy, nullable) NSString *placeholder;
@property (nonatomic, strong, nullable) UIColor *placeholderColor;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,19 @@ - (BOOL)canPerformAction:(SEL)action withSender:(id)sender
return [super canPerformAction:action withSender:sender];
}

#pragma mark - Dictation

- (void)dictationRecordingDidEnd
{
_dictationRecognizing = YES;
}

- (void)removeDictationResultPlaceholder:(id)placeholder willInsertResult:(BOOL)willInsertResult
{
[super removeDictationResultPlaceholder:placeholder willInsertResult:willInsertResult];
_dictationRecognizing = NO;
}

#pragma mark - Placeholder

- (void)_invalidatePlaceholderVisibility
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ NS_ASSUME_NONNULL_BEGIN
@property (nonatomic, copy, nullable) NSString *placeholder;
@property (nonatomic, strong, nullable) UIColor *placeholderColor;
@property (nonatomic, assign, readonly) BOOL textWasPasted;
@property (nonatomic, assign, readonly) BOOL dictationRecognizing;
@property (nonatomic, assign) UIEdgeInsets textContainerInset;
@property (nonatomic, strong, nullable) UIView *inputAccessoryView;
@property (nonatomic, strong, nullable) UIView *inputView;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,12 @@ - (BOOL)textOf:(NSAttributedString *)newText equals:(NSAttributedString *)oldTex
}
}];

BOOL shouldFallbackToBareTextComparison =
[self.backedTextInputView.textInputMode.primaryLanguage isEqualToString:@"dictation"] ||
BOOL shouldFallbackDictation = [self.backedTextInputView.textInputMode.primaryLanguage isEqualToString:@"dictation"];
if (@available(iOS 16.0, *)) {
shouldFallbackDictation = self.backedTextInputView.dictationRecognizing;
}

BOOL shouldFallbackToBareTextComparison = shouldFallbackDictation ||
[self.backedTextInputView.textInputMode.primaryLanguage isEqualToString:@"ko-KR"] ||
self.backedTextInputView.markedTextRange || self.backedTextInputView.isSecureTextEntry ||
fontHasBeenUpdatedBySystem;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ NS_ASSUME_NONNULL_BEGIN
@property (nonatomic, assign) BOOL caretHidden;
@property (nonatomic, assign) BOOL contextMenuHidden;
@property (nonatomic, assign, readonly) BOOL textWasPasted;
@property (nonatomic, assign, readonly) BOOL dictationRecognizing;
@property (nonatomic, strong, nullable) UIColor *placeholderColor;
@property (nonatomic, assign) UIEdgeInsets textContainerInset;
@property (nonatomic, assign, getter=isEditable) BOOL editable;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,19 @@ - (BOOL)canPerformAction:(SEL)action withSender:(id)sender
return [super canPerformAction:action withSender:sender];
}

#pragma mark - Dictation

- (void)dictationRecordingDidEnd
{
_dictationRecognizing = YES;
}

- (void)removeDictationResultPlaceholder:(id)placeholder willInsertResult:(BOOL)willInsertResult
{
[super removeDictationResultPlaceholder:placeholder willInsertResult:willInsertResult];
_dictationRecognizing = NO;
}

#pragma mark - Caret Manipulation

- (CGRect)caretRectForPosition:(UITextPosition *)position
Expand Down

0 comments on commit e8b4bb0

Please sign in to comment.