Lafina AI Assistant 🤖

Lafina is a privacy-focused, 100% on-device AI secretary built with React Native. It uses local LLMs and Speech-to-Text models to help you manage your calendar through natural voice commands and simulated incoming calls.

Key Features

• On-Device LLM: Powered by SmolLM2-135M via llama.rn.
• Offline STT: Powered by Whisper Tiny via whisper.rn.
• Pre-crafted Responses: Context-aware reminder scripts generated during task creation.
• In-Call Interaction: "Barge-in" UI that lets you talk to the AI while simulate-calling.
• Background Persistence: Checks for scheduled tasks even when the app is closed.

---

📚 Documentation Checklist

This section outlines all necessary documentation for the project, including current implementations and future roadmap items.

1. Project Overview & Setup

• Installation Guide
  • Prerequisites (Node.js, JDK, Android Studio/Xcode, Watchman)
  • Environment Setup (Android ANDROID_HOME & iOS CocoaPods)
  • Critical: Model Assets Download Links & Placement (assets/models/)
  • Step-by-step generic "Build & Run" commands
• Project Structure / Architecture Overview
  • Explanation of src/services (Singleton patterns: DatabaseService, LLMService, VoiceService)
  • Explanation of src/screens vs src/components vs src/navigation
  • State Management philosophy (zustand stores)
  • Data Flow Diagram (Audio -> STT -> LLM -> TTS)

2. Core Service Documentation

• AI & LLM Service (LLMService.ts)
  • Configuration options for llama.rn (Context size, Threads, GPU/NPU delegates)
  • Prompt Engineering Templates (System Prompts used for specific tasks)
  • Benchmarks (Inference speed on target devices)
• Voice Pipeline (VoiceService.ts / VADService.ts)
  • VAD Threshold Tuning Guide (How to adjust sensitivity)
  • STT Model nuances (whisper.rn quantization trade-offs)
  • TTS Engine configuration (System voice vs Kokoro ONNX)
• Database & Persistence (DatabaseService.ts)
  • SQLite Schema Documentation (Tables: tasks, events, logs)
  • Migration Strategy (How to handle schema updates)
  • op-sqlite optimization settings
• Background Tasks (BackgroundService.ts)
  • OS-specific constraints (Android Battery Optimization / iOS Background Fetch)
  • react-native-background-actions configuration
  • Notification Channel setup

3. Feature Specifics

• Call Simulation (CallManager.ts)
  • State Machine diagram (Idle -> Ringing -> Active -> Ended)
  • react-native-callkeep integration details (Native UI vs Custom UI)
  • Audio Session Management (How to handle interruptions)
• Smart Calendar (CalendarScreen.tsx)
  • react-native-calendars customization guide (Theming, Custom Renderers)
  • Agenda logic explanation
• Developer Tools
  • How to use DevConsoleScreen
  • Enabling/Disabling Logs in production

4. API & Interface Documentation (Internal)

• TypeScript Types
  • Documentation of Task, Event, ChatMessage interfaces
• Navigation Params
  • RootStackParamList definitions

5. Future Implementation Roadmap (To Be Documented When Implemented)

• Theming System
  • Design System tokens (Colors, Typography)
  • Dark/Light mode implementation details
• Onboarding & Permissions
  • Permission Flow implementation (Rationale > Request > Result)
• Testing Strategy
  • Unit Testing Guide (Jest)
  • E2E Testing Guide (Maestro/Detox)
• Release & Deployment
  • Android keystore management & Singing configs
  • ProGuard rules explanation
  • iOS Provisioning Profiles & TestFlight upload steps
• Cloud Sync (Planned)
  • Firestore Schema (Parallel to SQLite)
  • Sync Conflict Resolution Strategy

---

Prerequisites

Before you begin, ensure you have the React Native Environment set up for your OS.

• Node.js (v18+)
• Android Studio (for Android) / Xcode (for iOS)
• CocoaPods (iOS only)

---

Installation & Setup

1. Environment Setup

Follow the Official React Native Guide to set up your development environment for Android (primary) or iOS.

2. Automated Setup

We have created a script to automatically download all required models and link them to your project.

Run the following command:

npm run setup

This script will:

1. Create assets/models/ directory.
2. Download all required models (LLM, Whisper, VAD, Kokoro, Voices).
3. Automatically run npx react-native-asset to link them.

3. Manual Installation (Alternative)

If you prefer to download manually or npm run setup fails, you can download all model files from this Google Drive folder:

👉 Download All Models (Google Drive)

Directory Structure:

Lafina-AI/
├── assets/
│   └── models/
│       ├── SmolLM2-135M-Instruct.gguf
│       ├── ggml-tiny.en.bin
│       ├── silero_vad.onnx
│       ├── kokoro-v0_19.onnx
│       ├── af_bella.bin   <-- (If downloaded as 'voices.bin', RENAME to 'af_bella.bin')
│       └── cmudict.txt

Individual Source Links:

Component	Filename	Source / Download Link	Purpose
LLM	SmolLM2-135M-Instruct.gguf	HuggingFace (Q8_0)	Natural Language Processing & Intent Extraction
STT	ggml-tiny.en.bin	HuggingFace (Whisper)	Speech-to-Text Transcription
VAD	silero_vad.onnx	HuggingFace (Coqui Mirror)	Voice Activity Detection (Silent filtering)
TTS Model	kokoro-v0_19.onnx	HuggingFace (thewh1teagle)	Text-to-Speech Synthesis
TTS Style	af_bella.bin	HuggingFace (ONNX Community)	Specific Voice Style (Bella)
Dictionary	cmudict.txt	GitHub (CMU Source)	Phoneme dictionary for TTS

Then Link Assets:

npx react-native-asset

3. Build & Run

1. Clone and Install Dependencies:

   npm install
   # OR
   yarn install

2. Link Assets: Run this to bundle the models into the native application:

   npx react-native-asset

3. Android Specifics: Open the /android folder in Android Studio and wait for Gradle sync to complete. If you encounter the VersionNumber error, see Troubleshooting.

4. iOS Specifics:

   cd ios && pod install && cd ..

5. Start Metro & Run:

   # Terminal 1
   npm start
   
   # Terminal 2
   npm run android  # OR npm run ios

---

Architecture Overview

• ConversationService.ts: The orchestrator. Fast-tracks audio -> text -> intent -> reply.
• LLMService.ts: Manages the llama.rn context and system prompting.
• AudioService.ts: Wraps Recording and whisper.rn transcription.
• VoiceService.ts: Abstracted Voice Engine (supports System TTS and future Kokoro ONNX).
• CallManager.ts: Handles incoming call simulation and in-call audio routing.
• DatabaseService.ts: Local storage via op-sqlite.

---

Troubleshooting

Android Build Fails with VersionNumber error

If the build fails with Could not get unknown property 'VersionNumber', it's because onnxruntime-react-native uses a deprecated Gradle API. Fix: Open node_modules/onnxruntime-react-native/android/build.gradle and comment out the following block (around line 250):

// if (VersionNumber.parse(REACT_NATIVE_VERSION) < VersionNumber.parse("0.71")) {
//   extractLibs "com.facebook.fbjni:fbjni:+:headers"
//   extractLibs "com.facebook.fbjni:fbjni:+"
// }

Native Module "NativeVAD" Not Found

Ensure you have rebuilt the app with npm run android after adding the Kotlin files. If the error persists, check MainApplication.kt to ensure VADPackage() is added to the list.

• Android 14+ Permissions: Ensure ACCESS_FINE_LOCATION and POST_NOTIFICATIONS are granted if background triggers fail.
• Model Loading: If the app crashes on start, verify that the .gguf file exists in src/assets/models/ and was correctly linked via react-native-asset.
• Audio Conflicts: If you hear no sound during calls, check the CallManager logs to ensure InCallManager started correctly.

---

License

Private Project - Lafina-AI Team.