Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][iOS]Add TextField OCR engine side support #29229

Closed
wants to merge 21 commits into from

Conversation

luckysmg
Copy link
Contributor

@luckysmg luckysmg commented Oct 18, 2021

Preview:
https://user-images.githubusercontent.com/49340347/137671033-cf99050f-e4f3-47eb-a346-1b7e4063a2b8.mp4

Related PR in flutter framework (WIP)
flutter/flutter#96637

List which issues are fixed by this PR. You must list at least one issue.
flutter/flutter#91896

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
  • I listed at least one issue that this PR fixes in the description above.
  • I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
    writing and running engine tests.
  • I updated/added relevant documentation (doc comments with ///).
  • I signed the CLA.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@luckysmg
Copy link
Contributor Author

The new API captureTextFromCamera is available on Xcode 13,Xcode 12 doesn't have this API

@jmagman
Copy link
Member

jmagman commented Oct 18, 2021

The new API captureTextFromCamera is available on Xcode 13,Xcode 12 doesn't have this API

This is dependent on flutter/flutter#85555

@luckysmg luckysmg marked this pull request as draft November 19, 2021 09:13
@luckysmg luckysmg mentioned this pull request Nov 20, 2021
8 tasks
@luckysmg
Copy link
Contributor Author

I want to use ARBodyTrackingConfiguration.isSupported in engine to get an bool value whether this device support OCR?
But raise some error when engine compile...

@flutter-dashboard flutter-dashboard bot added the platform-web Code specifically for the web engine label May 27, 2022
@luckysmg luckysmg changed the title Support iOS TextField OCR on iOS 15 [WIP][iOS]Support iOS TextField OCR on iOS 15 May 27, 2022
@luckysmg luckysmg changed the title [WIP][iOS]Support iOS TextField OCR on iOS 15 [iOS]Support iOS TextField OCR on iOS 15 May 28, 2022
@luckysmg luckysmg marked this pull request as ready for review May 28, 2022 05:40
@luckysmg luckysmg changed the title [iOS]Support iOS TextField OCR on iOS 15 [iOS]Add TextField OCR engine side support May 28, 2022
@luckysmg
Copy link
Contributor Author

Hi @chunhtai Could you give this PR a review? Thx😄

Copy link
Member

@jmagman jmagman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, @LongCatIsLooong would you mind also taking a look?

Copy link
Contributor

@LongCatIsLooong LongCatIsLooong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -1621,6 +1634,11 @@ - (NSString*)contrastMode {
}
}

- (BOOL)captureTextFromCameraEnabled {
UITextField* textField = [self textField];
return [textField canPerformAction:@selector(captureTextFromCamera:) withSender:nil];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a notification you can listen to, if the value you get here changes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For instance will the value become false, if the user rejects the camera permission?

Copy link
Contributor Author

@luckysmg luckysmg Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This api invokes system internal camera API and it seems that is will not ask users for camera permission..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iOSDemo 2.zip
Here is demo code. I didn't request any permission

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the capability of monitoring the results of this canPerformAction: call I think it would be better to implement this feature in a way that's similar to paste: introduce an API that allows the developer to query whether live text is currently enabled on the platform, otherwise they may get outdated value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I will take a look. Thx

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I thought you could still turn camera access off in Settings? (don't have a test device available atm, I'll give this a try tomorrow)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah if you disable camera access in Settings -> Screen Time -> Content & Privacy Restrictions -> Allowed Apps, the text edit menu item will be gone.

@luckysmg
Copy link
Contributor Author

luckysmg commented Jun 1, 2022 via email

@luckysmg luckysmg changed the title [iOS]Add TextField OCR engine side support [WIP][iOS]Add TextField OCR engine side support Jun 14, 2022
@luckysmg luckysmg marked this pull request as draft June 14, 2022 15:06
@luckysmg
Copy link
Contributor Author

Hi @jmagman @LongCatIsLooong
I will close this PR. And I raise a new PR with a better solution.Please review that PR. Thx😄
New PR link: #34751

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes platform-ios platform-web Code specifically for the web engine
Projects
None yet
3 participants