Your move. Pay up.
CheckMate is a native Android bill-splitting app that scans restaurant receipts, extracts itemized data using on-device and cloud AI, and splits the bill fairly between people — all without a cloud backend or user accounts.
Point your camera at a receipt. CheckMate reads it, extracts every item, applies tax proportionally, and lets you assign dishes to people. Each person sees exactly what they owe. Share via WhatsApp or scan a DuitNow QR code to collect payment.
Everything stays on your device. No account. No server. No subscription. Unless you choose to use Cloud AI, you would need an API key for OpenRouter / Anthropic The AI part and implementation is still under fine-tuning and improvement
- CameraX live viewfinder with tap-to-focus
- Gallery picker for existing photos
- uCrop integration for crop and rotate
- ML Kit OCR with spatial layout reconstruction
- Grayscale + contrast preprocessing for better OCR accuracy
The pipeline runs in a tiered fallback chain — each tier is tried in order until confidence is sufficient:
| Tier | Provider | Requires |
|---|---|---|
| 1 | Gemini Nano (AICore) | Pixel 8+ or Galaxy S24+ |
| 2 | LiteRT on-device model | 6GB+ RAM, downloadable |
| 3 | Rule-based OCR parser | Always available |
| 4 | OpenRouter (cloud) | API key in Settings |
The rule-based parser handles Malaysian receipts without any LLM — it uses spatial zone detection, a 3-line item pattern (name → quantity tag → price), and arithmetic verification to extract items reliably on any Android device.
- All fields editable before saving
- Amber highlights on low-confidence fields
- Per-field confidence indicators
- Arithmetic mismatch detection
- Toggle between SST-inclusive and SST-exclusive
- Add people from saved contacts or as one-time guests
- Designate who paid
- Assign items per person (tap to assign, long-press for multi-select)
- Equal split shortcut
- Tax auto-calculated proportionally by item subtotal
- Partial payment tracking
- Person cards with itemized breakdown (collapsed/expanded)
- Settlement toggle per person
- Partial amount received
- DuitNow QR tab
- Touch 'n Go eWallet tab
- Bank transfer tab
- WhatsApp share: per-person summary or full group summary
- Swipe to delete
- Tap card to view detail
- Filter by category
Connect your OpenRouter API key in Settings to use free vision-capable models:
| Model | Context | Notes |
|---|---|---|
| Mistral Small 3.1 24B | 128K | Best JSON schema adherence; default |
| NVIDIA Nemotron Nano 12B VL | 128K | Vision-language specialist |
| Google Gemma 3 27B | 131K | Best accuracy for complex receipts |
| Google Gemma 3 12B | 32K | Balanced speed/accuracy |
| Google Gemma 3 4B | 32K | Fastest; simple receipts |
When OpenRouter is configured, the pipeline sends both the receipt image and the ML Kit OCR text to the model for highest accuracy. Falls back to the rule-based parser if rate-limited.
| Layer | Technology |
|---|---|
| Language | Kotlin |
| UI | Jetpack Compose |
| Architecture | MVVM + Hilt |
| Database | Room (SQLite) |
| Camera | CameraX |
| OCR | ML Kit Text Recognition v2 |
| Image crop | uCrop |
| On-device AI | Gemini Nano via AICore |
| Cloud AI | OpenRouter API |
| API key storage | EncryptedSharedPreferences |
| Settings storage | DataStore |
| Dependency injection | Hilt |
| Image preprocessing | custom ImagePreprocessor (grayscale, contrast, 800px scale) |
- Android 12 (API 31)
- Target SDK 35
- Gemini Nano: Android 14+ on Pixel 8 / Galaxy S24 series
- OpenRouter cloud features: internet connection + free API key
app/src/main/java/com/kishankathi/checkmate/
│
├── ai/
│ ├── AIProvider.kt ← Provider interface
│ ├── AIProviderFactory.kt ← Tier routing + Nano probe
│ ├── AIResult.kt
│ ├── GeminiNanoProvider.kt ← Gemini Nano via AICore
│ ├── MLKitOcrExtractor.kt ← Spatial OCR with zone detection
│ ├── MLKitProvider.kt ← Rule-based fallback (no LLM)
│ ├── OpenRouterProvider.kt ← Vision + OCR hybrid cloud call
│ ├── OpenRouterModelCatalogue.kt ← 5 free model definitions
│ ├── ImagePreprocessor.kt ← Grayscale, contrast, resize
│ ├── ReceiptLayoutParser.kt ← Rule-based item extractor
│ ├── ReceiptJsonParser.kt ← JSON → domain models
│ └── ReceiptJsonValidator.kt ← Arithmetic verification
│
├── data/
│ ├── db/
│ │ ├── AppDatabase.kt
│ │ ├── entities/ ← Receipt, ReceiptItem, Person, Split
│ │ └── dao/
│ └── prefs/
│ └── CloudApiPreferences.kt ← EncryptedSharedPrefs + DataStore
│
├── ui/
│ ├── screens/
│ │ ├── camera/ ← CameraX viewfinder
│ │ ├── review/ ← Editable AI review screen
│ │ ├── people/ ← Add people + payer selection
│ │ ├── assign/ ← Per-item assignment
│ │ ├── summary/ ← Person cards + settlement
│ │ ├── history/ ← Receipt list
│ │ ├── settings/ ← AI provider + OpenRouter config
│ │ └── onboarding/ ← Name setup
│ └── theme/ ← Design tokens
│
└── MainActivity.kt
| Token | Value | Usage |
|---|---|---|
| BackgroundPrimary | #0D0D12 |
App background |
| SurfaceDefault | #141418 |
Cards |
| SurfaceElevated | #1C1C24 |
Elevated surfaces |
| AccentGreen | #00C896 |
Primary actions, confidence indicators |
| AccentAmber | #F5A623 |
Warnings, fields needing review |
| AccentRed | #E8503A |
Errors, delete actions |
| TextPrimary | #F0EFE9 |
Main text |
| TextSecondary | #8B8B96 |
Supporting text |
| TextMuted | #55555E |
Hints, labels |
ML Kit returns text blocks with bounding boxes. The extractor sorts blocks by Y position, classifies each line into a spatial zone (HEADER / ITEMS / TOTALS), and reconstructs a structured text layout:
=== HEADER ===
Andra by Gula Cakery. Eco Ardence
Lab 12. Ardence Labs, 40170 Setia Alam
=== ITEMS ===
ITEM: Minty Lime Sparkler | QTY: 1x S6 | PRICE: 16.00
ITEM: Grilled Chicken Chop | QTY: 1x S6 | PRICE: 35.00
=== TOTALS ===
Crunchy Taco Combo
2x S6
106.00
TAX_LINE: Service Charge (10%)
21.10
TAX_LINE: SST (6%)
12.66
Grand Total
RM244.75
Malaysian restaurant receipts often split each item across three lines:
Crunchy Taco Combo ← ITEM_NAME
2x S6 ← QTY_TAG (quantity + size code)
106.00 ← AMOUNT
The parser classifies each line and uses a lookahead to assemble items correctly, handling both 2-line (name → price) and 3-line (name → qty tag → price) patterns.
Tax lines are tagged during OCR reconstruction. The parser identifies amounts using a 4-strategy fallback:
- Explicit
AMOUNT:tag - Clean next-line amount
- Smallest amount extracted from fused OCR text (e.g.
R黑1.32 RM 21.95) - Garble-stripped regex
After extraction, the validator checks:
subtotal + tax_gst_sst + tax_service ≈ total (within RM 0.10)
If the difference exceeds the tolerance and both tax fields are zero, it attempts to back-calculate using Malaysian tax rates (SST 6%, service charge 10%). It skips back-calculation if the subtotal is zero to avoid assigning the full bill amount as tax.
The confidence score is computed deterministically, not guessed by the model:
| Check | Weight |
|---|---|
| Merchant name valid | 20% |
| Items extracted | 25% |
| Quantities realistic | 10% |
| Grand total non-zero | 20% |
| Arithmetic balance | 15% |
| Date/time extracted | 5% |
| Category specific | 5% |
- Android Studio Hedgehog or later
- Android SDK 35
- Kotlin 1.9+
- Java 17
git clone https://github.com/your-username/checkmate.git
cd checkmateOpen in Android Studio and sync Gradle. Build and run on a physical device (API 31+).
The camera does not work on emulators. Use a physical device for all testing.
- Create a free account at openrouter.ai
- Generate an API key from Settings → API Keys
- In CheckMate: Settings → AI Provider → OpenRouter → paste key → select model → Save
Free tier models have rate limits. The app falls back to the rule-based parser automatically if rate-limited.
All AI pipeline events are logged under the tag CheckMate_AI. Filter Logcat to this tag to see the full pipeline trace for any scan:
adb logcat -s CheckMate_AI
Key log markers:
[FACTORY]— which provider was selected and why[PREPROCESS]— image dimensions and compression[OCR]— block count, zone boundaries, tax line detection, full layout[RULE_PARSER]— merchant candidate, date found, items extracted per strategy[OPENROUTER]— request size, response code, token usage, raw JSON[PARSER]— JSON parsing, merchant sanitization, tax verification[VALIDATOR]— arithmetic check, flags added
CheckMate is built specifically for Malaysian receipts with support for:
- SST (Sales and Services Tax) — 6% government tax
- Service Charge — typically 10%, labeled "Servis", "Caj Perkhidmatan"
- DuitNow QR payment display
- Touch 'n Go eWallet payment display
- MYR currency formatting throughout
- Bahasa Malaysia tax labels in OCR detection (Cukai Perkhidmatan, Caj Perkhidmatan, Jumlah)
- Multi-language receipts — mixed English/Malay/Chinese text handled by ML Kit's multilingual model
- Gemini Nano is only available on Pixel 8+ and select Galaxy S24+ devices. All other devices use the rule-based parser or OpenRouter.
- Free OpenRouter models have upstream rate limits — scanning rapidly in succession will trigger 429 errors. The app falls back gracefully.
- Heavily damaged, handwritten, or dot-matrix receipts have lower accuracy ceilings regardless of AI tier.
- The
(S6size code on this receipt format (Eco Ardence / Ardence Labs venues) is an internal size tag, not a menu item — the parser correctly ignores it.
- Cloud sync / backup (opt-in, end-to-end encrypted)
- Batch scanning — scan multiple receipts from a folder
- Export: PDF expense report, CSV, JSON
- WhatsApp deep links with pre-filled payment amounts
- Recurring receipt detection
- Spending analytics by category and period
- Currency conversion for multi-currency receipts
- Natural language search ("show me food receipts over RM50 in January")
This project is currently in active development. Issues and pull requests are welcome once the codebase stabilizes.
MIT License. See LICENSE for details.
CheckMate — Your move. Pay up. ♟