Architecture

Architecture

Zerm is a single native macOS app (Swift/SwiftUI + AppKit) organized around one engine and three on-device AI models. This page is the map; each subsystem has its own page.

High-level overview

flowchart TB
    subgraph Input
        HK[HotkeyManager<br/>bare-modifier + KeyboardShortcuts]
        SEL[SelectedTextService<br/>reads on-screen selection]
        MIC[CoreAudioRecorder<br/>AUHAL microphone]
    end

    subgraph Core
        ENG["ZermEngine<br/>@MainActor orchestrator<br/>RecordingState machine"]
        UI[RecorderUIManager<br/>notch / mini widget]
    end

    subgraph Models[Three on-device models]
        STT[Whisper<br/>speech-to-text]
        TTS[Kokoro<br/>text-to-speech]
        LLM[Gemma<br/>llama.cpp]
    end

    subgraph Output
        ENH[AIEnhancementService]
        PASTE[CursorPaster<br/>paste at cursor]
        PLAY[TTSPlayer<br/>AVAudioEngine]
    end

    HK --> ENG
    MIC --> ENG
    ENG --> STT --> ENH --> PASTE
    ENG --> UI
    HK --> TTSC[TTSController]
    SEL --> TTSC
    TTSC --> SR{Smart Reading}
    SR -->|always| NORM[TTSTextNormalizer]
    SR -->|optional| LLM
    NORM --> TTS --> PLAY
    LLM --> TTS
    ENH -.optional.-> LLM
    TTSC --> UI

Loading

Two independent flows share one widget and one state machine:

• Dictation (left): microphone → Whisper → optional enhancement → paste.
• Read Aloud (right): selection → smart reading → Kokoro → playback.

They are mutually exclusive — a single RecordingState on ZermEngine is the source of truth, so dictation and Read Aloud can never run at once.

Tech stack

• Language/UI: Swift 5.10+, SwiftUI + AppKit (NSPanel overlays)
• Data: SwiftData (history, vocabulary, replacements)
• Audio: CoreAudio AUHAL (capture), AVFoundation/AVAudioEngine (playback)
• On-device AI: whisper.cpp, sherpa-onnx (+ ONNX Runtime), llama.cpp — all prebuilt XCFrameworks
• Updates: Sparkle 2.x · Packaging: Xcode + xcodebuild
• Key SPM deps: KeyboardShortcuts, LaunchAtLogin-Modern, Sparkle, FluidAudio, LLMkit, swift-atomics, Zip, SelectedTextKit, MediaRemoteAdapter

The recording state machine

RecordingState (on ZermEngine) gates everything. The hotkey is ignored unless the state allows it (canProcessHotkeyAction).

stateDiagram-v2
    [*] --> idle
    idle --> starting: dictation hotkey
    starting --> recording
    recording --> transcribing: stop
    transcribing --> enhancing: if AI enabled
    enhancing --> idle: paste
    transcribing --> idle: paste

    idle --> preparingSpeech: Read Aloud hotkey
    preparingSpeech --> generatingSpeech: AI rewrite on
    generatingSpeech --> preparingSpeech: rewrite done
    preparingSpeech --> speaking: first audio chunk
    speaking --> idle: finished / cancelled
    generatingSpeech --> idle: cancelled

Loading

The widget label is driven directly by this state: Transcribing / Enhancing (dictation) and Thinking… / Preparing… / live bars (Read Aloud).

Key shared objects

Class	Role
ZermEngine	Recording + pipeline orchestration; owns RecordingState
RecorderUIManager	Notch/mini widget lifecycle; Read Aloud ⇄ dictation arbitration
HotkeyManager	Bare-modifier NSEvent monitors + KeyboardShortcuts
TTSController	Read Aloud orchestrator (fetch → prepare → synth → play)
WhisperModelManager · KokoroModelManager · LocalLLMModelManager	The three model downloaders/loaders (same UX)
AIEnhancementService / AIService	LLM enhancement pipeline + provider routing
WindowManager	Main window + accessory-policy handling

Repository layout

The Xcode project lives at the repository root (it matches VoiceInk; it used to be nested under native-macos/ to coexist with a since-removed Tauri prototype).

Zerm.xcodeproj         Xcode project (app + tests)
Zerm/                  Swift source
  ZermEngine.swift     recording/transcription orchestrator
  HotkeyManager.swift  TextToSpeech/   Read Aloud subsystem
  LocalLLM/            on-device LLM (llama.cpp bridge)
  Transcription/       STT engines + pipeline
  Views/               SwiftUI UI (incl. Recorder widget)
  Services/            settings, model mgmt, integrations
Makefile               local build + ad-hoc install
docs/                  GitHub Pages site
Notebook/              Zettelkasten project notes

Native dependencies (XCFrameworks)

All three live under $(HOME)/Zerm-Dependencies/ and are linked (some embedded) by the Xcode project. A notable gotcha: whisper.cpp and llama.cpp both vendor ggml at different versions — importing both ggml-bearing Clang modules into Swift collides. Zerm isolates llama behind an Objective-C++ bridge so its ggml headers never reach Swift. See On-Device LLM.

Framework	Used by	Linkage
whisper.xcframework	Whisper STT	embed + sign
sherpa-onnx + onnxruntime	Kokoro TTS	static link
llama.xcframework	Gemma LLM	embed + sign (via Obj-C++ bridge)

See also: The Three-Model Platform · Build & Release