Architecture

Architecture

Draggy is an Electron app: a Node main process that owns everything privileged, and a React renderer that owns everything visible. The interesting decisions are mostly about the line between them.

The two processes

electron/     main process
  main.cjs          windows, protocols, every IPC handler
  preload.cjs       the entire surface the renderer can reach
  storage.cjs       chats and settings, SQLite
  library.cjs       the document index, SQLite
  search.cjs        web search providers and fallback order
  documents.cjs     reading and writing Word, Excel, PowerPoint
  runner.cjs        running Python and JavaScript
  adblocker.cjs     filtering for the embedded browser
  updater.cjs       checking, downloading, installing
  platform.cjs      the per-OS parts: VRAM detection, installers

src/          renderer
  agent/            the streaming tool-calling loop
  tools/            tool definitions and the registry
  voice/            capture, detection, turn-taking, speech

The renderer has no Node integration and no direct filesystem access. Everything it can do is a function in preload.cjs, and that file is deliberately short enough to read in one sitting. If something is not exposed there, the renderer cannot do it.

The renderer is served from a custom app:// protocol rather than file://, which gives it a real origin and lets a normal Content Security Policy apply.

Talking to Ollama

Directly, from the renderer, over 127.0.0.1:11434. It is a local HTTP service and routing it through IPC would buy nothing but a copy of every token.

src/ollama.ts holds the client: streaming NDJSON, model metadata, context sizing, pull progress. Pull progress is worth a look — Ollama reports per-layer byte counts that restart part way through, so the tracker accumulates by digest rather than reading the numbers off the latest line.

The agent loop

src/agent/agentLoop.ts. One turn is: build a system prompt from what is actually enabled, stream a reply, and if the model calls a tool, run it, append the result, and stream again — up to a cap.

Two paths through it. Models that support native tool calling get their tools in the request. Models that do not get a text catalogue in the prompt and their output is parsed for call syntax, which is what src/toolParsing.ts exists for.

The loop also handles running out of context mid-answer: src/agent/resume.ts works out how to continue without repeating the sentence the user already read.

Tools

src/tools/registry.ts is a registry keyed by name; src/tools/builtin.ts registers the actual ones. A tool declares its schema, whether it is available in the current environment, and how to run itself. The system prompt is built from whatever is registered and enabled, so turning a tool off removes it from the model's world rather than merely discouraging its use.

The voice pipeline

The part worth reading if you only read one.

microphone → AudioWorklet → VAD → gate → Whisper → model → chunker → voice

• capture.ts — an AudioWorklet at 16 kHz, so resampling happens once in native code. Muting is applied on the audio thread, not several frames later.
• vad.ts — Silero in a worker, with an energy detector as a fallback that keeps the same interface.
• gate.ts — turns speech probabilities into conversation events. It knows nothing about audio, models or output, which is why the whole turn-taking policy is unit tested without a microphone. Two thresholds rather than one, so a voice hovering at the boundary cannot rattle the state machine.
• turnDetector.ts — decides how long a pause has to be before your turn is over, from what you just said. A trailing "um" or a conjunction buys more time than a full stop.
• conversation.ts — the wiring, and the only file that knows about all of it.
• reply.ts — generation, including the search round trip. Held back only as long as the opening could still be a search marker, which is seven characters.
• chunker.ts — cuts streamed text at the earliest point that still sounds like a phrase, so synthesis starts before generation finishes.
• speaker.ts — sits between them and can suspend without discarding, which is what makes an interruption recoverable when it turns out to be "mhm".

Everything overlaps deliberately. Transcription starts during your pause rather than after it; the reply is spoken clause by clause as it is generated; the model is resident in memory before the first question.

Storage

SQLite through node:sqlite, no native module to rebuild. Chats, settings and the document index share one database in the app's user data folder.

Writes are debounced and coalesced per session, and flushed on quit. A failed write surfaces in the interface rather than being swallowed, because silently losing a conversation is worse than an ugly banner.

Tests

Vitest, around 670 of them, running in under two seconds because none of them touch a GPU, a network or a model. The parts that are hard to test — audio, inference, Electron itself — are pushed to the edges, and the parts that encode actual decisions are pure functions in the middle.

npm run check is typecheck, lint and tests together.