audio.cpp Reader — Android companion for audio.cpp Studio

Photograph a book page (or scan it), have it transcribed by a vision/OCR model on your PC, and hear it read back in one of your saved voices — a make-your-own audiobook reader.

Everything runs on your own machine. The phone is a thin client: it talks to one address — the audio.cpp Studio backend — which proxies to llama.cpp for OCR and to audiocpp_server for speech.

phone ──HTTP──▶ audio.cpp Studio (:8110) ──▶ llama.cpp (:8080)        page photo → text · chat · music prompts
                                          ──▶ audiocpp_server (:9090)  text → speech · speech → text · music

  

Features

• Read — reached from the Library's camera button, since a reading always starts either from a page you photograph or one you already saved. Capture with the document scanner (auto-crop/straighten, multi-page) or the manual camera (tap-to-focus, torch, shutter flash + haptic feedback, rapid multi-shot). OCR fills an editable text box per page.
• Playback — chunked so audio starts on the first chunk while the rest generate; follow-along highlights and scrolls to the page being read. Runs in the background with lock-screen / notification media controls (Media3), plus speed, sleep timer, and per-chunk skip.
• Library — saved readings live on the PC (shared with Studio) and can be exported for offline playback; offline entries are badged and playable with the PC off.
• Call — a spoken conversation. Talk, and a llama.cpp model answers aloud in your cloned voice. Hands-free by default (a pause ends your turn) or hold the orb to talk; tap it to interrupt. Pick the chat model from a list discovered on the PC, set the response length, and toggle thinking (its reasoning is shown, never spoken). Starting the call loads the models first, so the first reply isn't the slow one. Typing works too, for a noisy room. A call keeps running with the screen off — it moves into a notification with End and Mute, so you can put the phone in your pocket mid-conversation. Everything in the setup sheet is adjustable during a call; changes that can only affect the next answer say so. Mic sensitivity is a slider — drag right if your turns never start, left if the room keeps ending them. Conversations are kept only when you press Save, and a saved one can be continued later (or on the PC — the store is shared).
• Transcribe — speech to text from the microphone or a picked audio or video file. A video's audio track is extracted on the phone, so only the sound is uploaded, not the movie. The transcript can be copied, shared, read aloud in a chosen voice right there, or saved to the library — which turns a recorded lecture into a reading you can play back in a cloned voice.
• Music — write a line about the song you want, tap Enhance, and a llama.cpp model fills in the caption, lyrics, title, BPM, key and length; ACE-Step 1.5 on the PC renders it. Everything stays editable, structure tags ([verse], [chorus], …) go in at the caret, and up to four takes arrive one at a time. Each take keeps the request that made it, so Reproduce and Vary still work later. Takes stream through the same player as your readings, so they get the mini-player, lock-screen controls and the sleep timer — a three-minute track is ~33 MB and is not downloaded first.
• Voices — record a new voice (with a sample-passage prompt to read), saved to the same store the desktop uses.
• Settings — PC address, TTS model, OCR model (e.g. PaddleOCR-VL vs a general VLM), transcription model, capture mode, optional language. The Call and Music screens keep their own model choices, so picking a fast voice for conversation does not change the one your readings use.
• Free VRAM lives in the app bar, badged with what the PC is currently holding, and covers both inference servers — the phone can ask the desktop to let go of the GPU without walking over to it.

What the PC needs

1. Studio reachable on the LAN. config.toml must have [server] host = "0.0.0.0".
2. A firewall rule, once, from an elevated PowerShell:

   New-NetFirewallRule -DisplayName "audio.cpp Studio (LAN)" -Direction Inbound `
     -Action Allow -Protocol TCP -LocalPort 8110 -Profile Private

3. A llama.cpp OCR server running with a vision model from [[llama.ocr_model]] in Studio's config.toml.
4. Studio running (scripts\launch_desktop.bat). It starts audiocpp_server on connect if it is not already up.
5. For Transcribe, a speech-recognition model downloaded in audio.cpp (e.g. qwen3_asr_0_6b). ffmpeg on the PC is only needed for the rare file this phone cannot decode itself.
6. For Call, a chat model on the same llama.cpp server, and — for a call that feels like one — the streaming-capable pair: VoxCPM2 for speech and nemotron_asr_q8_0 for recognition (python tools/model_manager_v2.py install nemotron_asr_q8_0). Studio's [call] section picks them by default. Other models work, just a second or two slower per reply.
7. For Music, ACE-Step 1.5 installed in audio.cpp, plus a chat model on the llama.cpp server for the Enhance button.

Building

Android Studio, or from this directory (JDK 17):

$env:JAVA_HOME="C:\Java\jdk-17.0.11+9"
.\gradlew.bat assembleDebug
# app\build\outputs\apk\debug\app-debug.apk

Install over USB with .\gradlew.bat installDebug (or adb install -r app\build\outputs\apk\debug\app-debug.apk). Without a cable: open Studio on the phone and download the APK from its Telemetry tab, which serves whatever Gradle last built.

Any task that produces an installable artifact (assemble*, install*, bundle*) bumps versionCode in app/version.properties; compiling and the unit tests leave it alone. That is what stops an install from silently keeping the old app.

On first launch set the PC address in Settings (a home router's DHCP lease can move the Studio machine's IP between reboots).

Notes

• usesCleartextTraffic="true" is set: the backend is plain HTTP on the LAN. This is a trusted-home-network tool with no authentication — do not expose port 8110 beyond your own network.
• Recording is mono 16-bit WAV, matching the WAV-only upload endpoint.
• Chunking targets ~400 characters on sentence/paragraph boundaries (text/Chunker.kt, kept in sync with Studio's lib/chunk.ts). audiocpp_server is single-threaded, so chunks are generated one ahead of playback.
• minSdk 26, targetSdk 35. Media3 (ExoPlayer), CameraX, ML Kit document scanner, Jetpack Compose (Material 3).