Let TextRecognizer read non-Latin scripts - #5562
Merged
Merged
Conversation
The built-in OCR API had no way to say what writing system to read, so it was Latin-only: Android hard-coded ML Kit's Latin DEFAULT_OPTIONS and the Apple backend left VNRecognizeTextRequest on its default languages. Both frameworks can do better -- ML Kit ships a recognizer per script and Apple Vision takes recognitionLanguages -- the portable API just never exposed the choice. Reported in discussion #5555 for Japanese OCR. Adds com.codename1.ai.vision.TextScript with latin/chinese/devanagari/ japanese/korean selectors and VisionOptions.textScript(...): new TextRecognizer(new VisionOptions().textScript(TextScript.japanese())) The selector names a script rather than a language because that is what the recognizers are organized by, and it is the one shape both backends can honour. No existing constructor or default changes. Like VisionBackends, each selector call is also a build-time dependency marker, so a build carries only the models it asked for. On Android the adapters are compiled inside the generated app, so each script gets its own adapter source: one import per file is what lets the builder delete the models the app never selects. PlatformFeatureCatalog.Entry gained support for several required methods, so the iOS script pod lands only when the ML Kit backend AND that script are both selected. Apple Vision asks the OS which recognition languages it supports instead of hard-coding a per-release list, and fails the analysis with an explicit error when the script is not among them. Leaving the request on its defaults would have run Latin OCR over the page and returned confident nonsense; the error names mlKitTextRecognition() as the way out. Verified against the vendor artifacts rather than the docs alone: the four Java options classes and TextRecognition.getClient(TextRecognizerOptions- Interface) read out of the published AARs with javap, the pod subspec names out of the GoogleMLKit podspec, and the Vision selectors and their availability out of the Xcode SDK headers. The Android AI adapters are excluded from every build in this repo (they need ML Kit, which only the generated app has), so a new parity test ties the chain together instead: selector method -> retained adapter source -> class the port loads -> ML Kit artifact the catalog adds. A break there is otherwise silent, leaving CI green and the feature inert. Not exercised locally, both needing a device build: the iOS ML Kit branch and end-to-end recognition of real non-Latin text. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff50115486
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
|
Developer Guide build artifacts are available for download from this workflow run:
Developer Guide quality checks: |
Contributor
Cloudflare Preview
|
Contributor
✅ Continuous Quality ReportTest & Coverage
Static Analysis
Generated automatically by the PR CI workflow. |
Collaborator
Author
|
Compared 151 screenshots: 151 matched. Native Android coverage
✅ Native Android screenshot tests passed. Native Android coverage
Benchmark ResultsDetailed Performance Metrics
|
Collaborator
Author
|
Compared 181 screenshots: 181 matched. |
Both new capability failures -- Apple Vision having no recognition language for the selected script, and the ML Kit script model not being linked -- were reported as plain JSON errors, which IOSVisionImpl.parse classified as VisionException.BACKEND_ERROR. Android raises UNSUPPORTED for the same condition, so a caller could not use the portable code to decide whether the ML Kit fallback was worth offering and would have had to pattern-match on the English message. The native side now tags those two payloads with a symbolic errorCode and parse() maps it, so the condition is UNSUPPORTED on both platforms. An untagged failure still means a genuine backend error. Also fixes the three developer-guide Vale findings that turned CI red. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Compared 149 screenshots: 149 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
Collaborator
Author
|
Compared 148 screenshots: 148 matched. Benchmark Results
Detailed Performance Metrics
|
Collaborator
Author
|
Compared 217 screenshots: 217 matched. |
The iOS ML Kit path reached the per-script options class through NSClassFromString so that CN1Vision.m would keep compiling for a build that linked only some of the pods. That is the wrong way to buy it, and it would have shipped the feature inert. CocoaPods links these pods as static frameworks, and IPhoneBuilder passes -ObjC only for ads builds or the ios.objC hint. A class that no symbol references is therefore dropped at link time, so NSClassFromString would have returned nil -- and the analyzer reported "not linked into this build" -- on a build whose Podfile did include the model. The failure is silent in exactly the way a wrong ParparVM native name is: green build, dead feature. __has_include gives the same "compiles without the pod" property with a real symbol reference, which is what the neighbouring MLKitTextRecognition and MLKitBarcodeScanning blocks already do. It also gets the class names spell-checked by the compiler rather than at runtime on a device. Verified by compiling CN1Vision.m against the actual ML Kit frameworks pulled from their podspecs, in three configurations: all four script pods present, only the Japanese pod present (the realistic case, where the other three branches must compile out), and no ML Kit at all. A separate probe with #error in the else branch confirms all four header guards really fire rather than silently skipping, and that each options class and its init type-check against MLKCommonTextRecognizerOptions. The umbrella header and class names were read out of the downloaded frameworks, not assumed from the naming convention. No builder change is needed for any of this: the script pods reach the Podfile through the catalog's generic iosPods() loop, and INCLUDE_CN1_VISION keys off the analyzer classes, which any TextScript caller also references. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Compared 144 screenshots: 144 matched. |
Collaborator
Author
|
Compared 143 screenshots: 143 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the RFE in discussion #5555:
com.codename1.ai.vision.TextRecognizerhad no way to say what writing system to read, so it was Latin-only. Android hard-coded ML Kit's LatinTextRecognizerOptions.DEFAULT_OPTIONS, and the Apple backend leftVNRecognizeTextRequeston its default languages. Both frameworks can do better — ML Kit ships a recognizer per script, Apple Vision takesrecognitionLanguages— the portable API just never exposed the choice.The API
New
TextScriptwithlatin()/chinese()/devanagari()/japanese()/korean(), plusVisionOptions.textScript(...). It names a script, not a language: that is what the underlying recognizers are organized by, and it is the one shape both backends can honour. Each script model also reads the Latin text mixed into the same page.No existing constructor, signature, or default changes — an app that never names a script behaves exactly as before.
Per platform
recognitionLanguages, askingsupportedRecognitionLanguagesAndReturnError:what the OS actually supports rather than hard-coding a per-release list. When the OS supports none of them the analysis fails with an explicit error namingVisionBackends.mlKitTextRecognition()as the way out. Leaving the request on its defaults would have run Latin OCR over a Japanese page and returned confident nonsense.NSClassFromString, soCN1Vision.mstill compiles whichever pods a given build linked.Dependency selection
Like
VisionBackends, each selector call doubles as a build-time dependency marker, so a build carries only the models it asked for — a Japanese app does not ship the Korean or Devanagari models.The Android AI adapters compile inside the generated app, not in this repo, so each script gets its own adapter source: one
importper file is exactly what letspruneOptionalAiSourcesdelete the models the app never selects.PlatformFeatureCatalog.Entrygained support for several required methods (it held one), so the iOS script pod lands only when the ML Kit backend and that script are both selected — Apple Vision reads these scripts itself.Verification
Vendor facts were checked against the real artifacts, not the docs alone:
TextRecognition.getClient(TextRecognizerOptionsInterface)read out of the published AARs withjavapGoogleMLKitpodspecGreen locally: core-unittests (3 new), catalog tests (2 new), builder tests (5 new), SpotBugs at zero across ios/plugin/core-unittests, the CI quality-report script,
check-cast-semantics.sh, andcheck-native-signatures.sh— the native gained aStringargument, so its C name changed.CN1Vision.mwas syntax-checked against the iOS 26.2 SDK both with and withoutINCLUDE_CN1_VISION, and the Android adapters compiled against stubbed ML Kit APIs.A note on the new parity test. The Android AI adapters are excluded from every build in this repo, so a class-name typo there is not a compile error anywhere — CI stays green and the feature ships inert.
AndroidTextScriptParityTestties the chain together instead: selector method → retained adapter source → class the port loads → ML Kit artifact the catalog adds.Not verified locally
Both need a device build, and are worth confirming before release:
Deliberate tradeoff
The Latin model ships in every OCR app, including a Japanese-only one: the base adapter and the no-arg constructor both need it, and the catalog cannot express "unless a script was selected". Documented in the developer guide.
🤖 Generated with Claude Code