On-Device OCR (PP-OCRv6)

An Android app that performs fully on-device optical character recognition — no network calls for inference. Point it at an image and it detects text regions, recognizes the text in each region, draws the detected polygons over the image, and lets you tap regions to select and copy the recognized text.

What it does

• Pick an image from device storage.
• Run a two-stage OCR pipeline: text detection (find regions) → text recognition (read each region).
• Overlay detected polygons on the image; tap a region to select it.
• Copy selected text or copy all recognized text to the clipboard.
• Switch between a small bundled model and a larger, more accurate model.

OCR model: PP-OCRv6 (ONNX)

The app uses PaddleOCR PP-OCRv6 models exported to ONNX and run with ONNX Runtime for Android. OCR is two models per variant:

• Detection — DBNet-style text detector. ONNX input is fully dynamic [N, 3, H, W]. Preprocessing mirrors PaddleOCR DetResizeForTest (keep aspect ratio, longest side ≤ 960, each side rounded to a multiple of 32) with BGR + ImageNet normalization. Postprocessing is DBNet decode (thresh 0.3, box_thresh 0.6, unclip 1.5).
• Recognition — CTC recognizer. ONNX input is [N, 3, 48, W] (fixed height 48, dynamic width). Preprocessing keeps aspect ratio and normalizes pixel/127.5 - 1. The CTC vocabulary is ["blank"] + dict + [" "], where dict is read from the model's *_rec_inference.yml.

Two model variants

Variant	Size	Where the models live	Accuracy
Tiny	~6 MB total	Bundled in app/src/main/assets/models/	Lower
Medium	~138 MB	Downloaded on demand, or bundled (see build variants)	Higher

Medium models are pulled from the HuggingFace CDN on first use:

• Det: https://huggingface.co/PaddlePaddle/PP-OCRv6_medium_det_onnx/resolve/main/inference.onnx
• Rec: https://huggingface.co/PaddlePaddle/PP-OCRv6_medium_rec_onnx/resolve/main/inference.onnx

OCR works best on photos/scans of printed or screen text. Stylized UI fonts and pixel/bitmap fonts are harder for the models.

Tech stack

• Kotlin + Jetpack Compose (Material 3)
• Hilt for dependency injection
• ONNX Runtime Android 1.22.0

Project structure (com.xception.ppocr)

core/inference/       OnnxSessionManager, ImagePreprocessor, TextDetector, DbNetDecoder,
                      TextRecognizer, CtcDecoder, CharacterDictionary
data/                 OcrRepositoryImpl, ModelDownloader
di/                   Hilt modules (AppModule, DataModule)
domain/               models, OcrRepository, RecognizeTextUseCase
presentation/ocr/     OcrViewModel, OcrScreen, ImageOverlayCanvas, ModelSelectionBar

Build variants

The medium-model source is controlled by the LOAD_MODELS_FROM_ASSETS build flag:

Build type	LOAD_MODELS_FROM_ASSETS	Medium model source	Use for
stage	true	Bundled in assets/ (no download)	Testing medium models quickly
debug	false	Downloaded on demand from HuggingFace	Normal development
release	false	Downloaded on demand from HuggingFace	Shipping

How to run

Quick start (recommended)

A helper script builds, installs, launches, and tails OCR logs:

./build_install.sh            # stage variant — medium models load from assets, no download
./build_install.sh debug      # debug variant — medium models download on demand

Using the app

1. Tap to pick an image.
2. Choose Tiny or Medium (Medium prompts to download on debug/release the first time; stage has them bundled).
3. Run OCR. Detected regions are outlined on the image.
4. Tap regions to select them, then Copy Selected, or use Copy All.

Notes

• Detection currently uses axis-aligned bounding boxes; tilted text may include extra background. A rotated min-area-rect crop is a planned improvement.