Architecture

Architecture

Draggy is an Electron app: a Node main process that owns everything privileged, and a React renderer that owns everything visible. The interesting decisions are mostly about the line between them.

The two processes

electron/     main process
  main.cjs          windows, protocols, every IPC handler
  preload.cjs       the entire surface the renderer can reach
  storage.cjs       chats and settings, SQLite
  library.cjs       the document index, SQLite
  search.cjs        web search providers and fallback order
  documents.cjs     reading and writing Word, Excel, PowerPoint
  runner.cjs        running Python and JavaScript
  adblocker.cjs     the uBlock Origin engine and its filter lists
  favicon.cjs       site icons for search results, fetched and cached
  filters/          one supplemental filter list, shipped with the app
  updater.cjs       checking, downloading, installing
  platform.cjs      the per-OS parts: VRAM detection, installers
  appData.cjs       adopting the data folder left by an earlier app name
  logger.cjs        the rotating log file behind Settings -> Data

src/          renderer
  agent/            the streaming tool-calling loop
  tools/            tool definitions and the registry
  voice/            capture, detection, turn-taking, speech
  BrowserBar.tsx    the toolbar above a page you opened

The renderer has no Node integration and no direct filesystem access. Everything it can do is a function in preload.cjs, and that file is deliberately short enough to read in one sitting. If something is not exposed there, the renderer cannot do it.

The renderer is served from a custom app:// protocol rather than file://, which gives it a real origin and lets a normal Content Security Policy apply.

Two sessions

The second boundary, and the one that decides whether the browser works at all.

Draggy's own windows run on Electron's default session, where a strict Content Security Policy is applied to every response: default-src 'self' app: draggy:, frame-src 'none', form-action 'none'.

Every external page runs on a separate persistent partition with no policy of ours attached. This is not a detail. When both shared one session, that policy was being applied to other people's sites as well, which blocked their scripts, stylesheets, images, video, iframes and form submissions — the browser rendered wreckage and it was not obvious why.

Your browsing and the model's page reads deliberately share that partition, so a verification check you pass by hand leaves a cookie the model's next fetch can use. It also means one ad-blocker switch governs both, which is the behaviour people expect from a switch labelled "ad blocker".

The browser window

BaseWindow with two WebContentsViews: the toolbar, drawn by the renderer so it is the app's own typeface and colours, and the page below it.

A view clips what it draws to its own bounds, so a menu hanging below a 48-pixel toolbar is simply cut off. While a menu is open the toolbar view is grown to fit it and sits above the page; the part of that region the menu does not use is transparent, and clicking it closes the menu.

Draggy does not attempt to pass bot-detection challenges. It recognises one, stops, and says so — see Browser and Ad Blocking.

Talking to Ollama

Directly, from the renderer, over 127.0.0.1:11434. It is a local HTTP service and routing it through IPC would buy nothing but a copy of every token.

src/ollama.ts holds the client: streaming NDJSON, model metadata, context sizing, pull progress. Pull progress is worth a look — Ollama reports per-layer byte counts that restart part way through, so the tracker accumulates by digest rather than reading the numbers off the latest line.

The agent loop

src/agent/agentLoop.ts. One turn is: build a system prompt from what is actually enabled, stream a reply, and if the model calls a tool, run it, append the result, and stream again — up to a cap.

Two paths through it. Models that support native tool calling get their tools in the request. Models that do not get a text catalogue in the prompt and their output is parsed for call syntax, which is what src/toolParsing.ts exists for.

The loop also handles running out of context mid-answer: src/agent/resume.ts works out how to continue without repeating the sentence the user already read.

Tools

src/tools/registry.ts is a registry keyed by name; src/tools/builtin.ts registers the actual ones. A tool declares its schema, whether it is available in the current environment, and how to run itself. The system prompt is built from whatever is registered and enabled, so turning a tool off removes it from the model's world rather than merely discouraging its use.

The voice pipeline

The part worth reading if you only read one.

microphone → AudioWorklet → VAD → gate → Whisper → model → chunker → voice

• capture.ts — an AudioWorklet at 16 kHz, so resampling happens once in native code. Muting is applied on the audio thread, not several frames later.
• vad.ts — Silero in a worker, with an energy detector as a fallback that keeps the same interface.
• gate.ts — turns speech probabilities into conversation events. It knows nothing about audio, models or output, which is why the whole turn-taking policy is unit tested without a microphone. Two thresholds rather than one, so a voice hovering at the boundary cannot rattle the state machine.
• turnDetector.ts — decides how long a pause has to be before your turn is over, from what you just said. A trailing "um" or a conjunction buys more time than a full stop.
• conversation.ts — the wiring, and the only file that knows about all of it.
• reply.ts — generation, including the search round trip. Held back only as long as the opening could still be a search marker, which is seven characters.
• chunker.ts — cuts streamed text at the earliest point that still sounds like a phrase, so synthesis starts before generation finishes.
• speaker.ts — sits between them and can suspend without discarding, which is what makes an interruption recoverable when it turns out to be "mhm".

Everything overlaps deliberately. Transcription starts during your pause rather than after it; the reply is spoken clause by clause as it is generated; the model is resident in memory before the first question.

Storage

SQLite through node:sqlite, no native module to rebuild. Chats, settings and the document index share one database in the app's user data folder.

Writes are debounced and coalesced per session, and flushed on quit. A failed write surfaces in the interface rather than being swallowed, because silently losing a conversation is worse than an ugly banner.

Tests

Vitest, around 680 of them, running in under two seconds because none of them touch a GPU, a network or a model. The parts that are hard to test — audio, inference, Electron itself — are pushed to the edges, and the parts that encode actual decisions are pure functions in the middle.

npm run check is typecheck, lint and tests together.