OceanScribe

OceanScribe is a fully local desktop voice keyboard for Windows and Linux/X11. It streams microphone audio through NVIDIA Nemotron ASR, shows the live transcript in a compact overlay, and inserts the completed text into the application that was active during dictation.

Optional AI cleanup runs locally through a separate llama.cpp worker after the recording has finished. Cleanup is disabled by default while its model quality is still being evaluated. OceanScribe does not use cloud inference, accounts, telemetry, transcript history, or a local HTTP service.

OceanScribe is under active development. The current project version is 0.1.0; interfaces, model choices, and user-facing behavior may still change.

Architecture

The controller and both inference runtimes run in separate processes because NeMo-Speech.cpp and llama.cpp use independently versioned GGML revisions:

oceanscribe (Dear ImGui UI and controller)
  |-- JSONL pipes --> oceanscribe-asr-worker
  |                   `-- NeMo-Speech.cpp + its GGML
  `-- JSONL pipes --> oceanscribe-rewrite-worker
                      `-- llama.cpp + its GGML

The ASR worker owns the microphone. Audio is never persisted or transferred to the controller. Standard output is reserved for versioned JSONL protocol messages and worker diagnostics go to standard error.

Prerequisites

• CMake 3.26 or newer
• Ninja (recommended)
• a C++20 compiler
• Git
• Windows: Visual Studio 2022 Build Tools with the C++ workload and vcpkg
• Linux/X11: audio, OpenGL, X11, and XTEST development libraries
• optional: a compatible CUDA toolkit
• Hugging Face CLI for downloading the local models

Clone the repository and all nested submodules:

git clone --recurse-submodules https://github.com/Danmoreng/OceanScribe.git
cd OceanScribe

For an existing checkout:

git submodule update --init
git -C third_party/NeMo-Speech.cpp submodule update --init ggml

Build and test

Linux CPU build:

./scripts/build.sh
./scripts/test.sh

Build without the X11 UI using ./scripts/build.sh --skip-ui. Enable CUDA for both inference workers using ./scripts/build.sh --cuda.

Windows CPU build from PowerShell:

.\scripts\build.ps1
ctest --test-dir build\core --output-on-failure

Pass -Cuda for CUDA or -SkipUi for a worker-only build. The Windows build script detects a Visual Studio-bundled vcpkg installation automatically; VCPKG_ROOT remains available for a separate checkout.

Build products are written below build/:

build/asr-worker/oceanscribe-asr-worker
build/rewrite-worker/bin/oceanscribe-rewrite-worker
build/core/oceanscribe

On Windows the executables have an .exe suffix. NeMo is configured and installed independently under build/nemo-sdk and build/nemo-install; it is not added to the llama.cpp CMake tree.

Models

OceanScribe discovers models in the standard Hugging Face cache. The cache follows HF_HUB_CACHE, then HF_HOME/hub, then XDG_CACHE_HOME/huggingface/hub, with the usual per-user cache as fallback.

The ASR worker uses NVIDIA Nemotron 3.5 ASR Streaming 0.6B. Download its GGUF files with:

hf download nvidia/nemotron-3.5-asr-streaming-0.6b

The optional cleanup worker currently uses exactly gemma-4-E2B-it-Q4_0.gguf:

hf download ggml-org/gemma-4-E2B-it-GGUF gemma-4-E2B-it-Q4_0.gguf

OCEANSCRIBE_ASR_MODEL and OCEANSCRIBE_REWRITE_MODEL can override model paths during development. The UI intentionally has no arbitrary model picker. Model files are not source artifacts and retain their own licenses.

Usage

Windows

Start the tray application after a normal build:

.\build\core\oceanscribe.exe

Press Alt+R in a text field to start dictation. Press the configured shortcut or Enter to finish, and Escape to cancel. OceanScribe keeps its overlay topmost without making it the insertion target. When safe direct insertion is not possible, the completed text is copied to the clipboard for manual paste.

The tray menu starts or stops dictation, opens Settings and the in-memory pipeline debugger, changes the language, or quits the application.

Settings are stored in %LOCALAPPDATA%\OceanScribe\settings.json.

Linux/X11

Start the background application with:

./scripts/run-ui.sh

Alt+R starts or stops dictation, Enter finishes, Escape cancels, and Ctrl+Alt+Q exits. The X11 host uses the local clipboard and XTEST to paste into the most recent external focus target. Wayland support is not implemented.

Settings are stored in $XDG_CONFIG_HOME/oceanscribe/settings.json, or ~/.config/oceanscribe/settings.json when XDG_CONFIG_HOME is unset. The 100%, 125%, and 150% overlay scale presets enlarge the overlay text, controls, spacing, and window together without changing the Settings window.

AI cleanup

Cleanup defaults to Off. Selecting AI cleanup starts the rewrite worker and loads Gemma on demand. After recording stops, the complete final transcript is sent once for constrained cleanup. The result is shown for review before the user confirms insertion. Any model, validation, timeout, or worker failure falls back to the complete raw ASR transcript.

Gemma uses its GGUF-provided Jinja chat template with thinking disabled, greedy decoding, grammar-constrained JSON output, protected technical literals, and language-independent structural validation. See docs/CLEANUP_MODEL_STATUS.md for the current development status.

Protocol and development documentation

Both workers communicate through JSON Lines on stdin/stdout. See docs/PROTOCOL.md and docs/ARCHITECTURE.md.

Current implementation details and dependency pins are recorded in docs/IMPLEMENTATION_NOTES.md. The Windows diagnostic UI is described in docs/PIPELINE_DEBUGGER.md.

License

OceanScribe is licensed under Apache License 2.0. Third-party components and model artifacts retain their own licenses; see THIRD_PARTY_NOTICES.md.