Skip to content

1.14 Obsidian Vault Guide

Jump to bottom
Frédéric Clavert edited this page Jul 24, 2026 · 10 revisions

Obsidian Vault Guide

New in v1.0.0-rc.2. ClioDeck can point at an Obsidian vault folder, index the notes inside it, and treat them as a first-class RAG source alongside your PDFs (secondary) and Tropy archives (primary).

This integration was absorbed from the ClioBrain project during the RC2 fusion.

What you get

  • Indexing. ClioDeck walks the vault, parses markdown notes (frontmatter, wikilinks, tags), chunks them and stores embeddings in the shared .cliodeck/brain.db (obsidian_*-prefixed tables — folded in from the older standalone obsidian-vectors.db file, which no longer exists on a fresh project; see "Storage and migration" below).
  • Retrieval. Every Brainstorm turn pulls notes from the vault into its retrieval system prompt, with a citation back to the note path. The retrieval scope toggles in the chat panel let you mix biblio + primary + notes in any combination.
  • MCP exposure. The vault is queryable from external MCP clients via the search_obsidian tool on ClioDeck's MCP server (BM25 / FTS5).
  • Partial-success reporting. Indexing is never binary: the index report shows indexed / skipped / failed counts.

The vault is read-only from ClioDeck's point of view. Notes are not modified; nothing is written back to your .md files.

Linking a vault

Settings → Obsidian Vault (VaultConfigSection):

  1. Click Link a vault….
  2. Pick the vault folder — that's the folder Obsidian itself opens (the one containing the .obsidian/ directory, not a subfolder).
  3. The path appears in the status block.
  4. Click Index to start the first build.

The index is per-project. If you switch ClioDeck projects, the vault link doesn't follow — each project decides whether it wants a vault and which one.

Indexing

  • Index — incremental: only notes that changed since the last run are reprocessed.

  • Reindex — forces a full rebuild. Use it after you change embedding model in Settings → LLM, or if you suspect drift.

  • Unlink — drops the link to the vault folder. Re-linking the same folder triggers a full rebuild from scratch, not a reuse of the previous index.

    ⚠️ Real bug, verified against code: Unlink also deletes the vault's index file at the filesystem level (fs.unlink on obsidianStorePath() in fusion-handlers.ts) — but since the db consolidation below, that path is .cliodeck/brain.db, the file also holding your PDF vectors, Tropy primary sources, and research-journal history. Clicking Unlink does not just forget the vault link: it deletes the whole shared database for the project. Filed as issue #27.

While indexing runs, the section shows live progress (<stage> — <processed>/<total>).

⚠️ Real bug, verified against code: the fusion:vault:index handler captures the project path once at the start of the run (root in fusion-handlers.ts), so the indexing job itself always writes to the correct project's brain.db even if you switch projects mid-run — no data corruption there. But its progress channel, fusion:vault:progress, is a single unscoped Electron IPC event with no project identifier in the payload ({ stage, processed, total } only). Closing Settings mid-index, switching to a different project, and reopening Settings before the old job finishes attaches a new progress listener (for the new project) to the same channel — so you can see progress numbers from the old project's still-running job, misattributed to the project you're now looking at. Filed as issue #35.

When indexing finishes:

✓ Indexed: 412 · Skipped: 7 · Failed: 0

The Skipped count includes notes the parser intentionally left out (empty notes, frontmatter-only files, binary files masquerading as .md, etc.). Failed is for I/O or parse errors — investigate those.

Using the vault in Brainstorm

In the Brainstorm chat panel, the source-type toggles control which retrievers run:

Toggle combination Effect
Biblio only Secondary sources (PDFs) only.
Primary only Tropy archives only.
Notes only Vault only.
Biblio + Primary (no notes) PDFs + Tropy.
Biblio + notes PDFs + vault.
Primary + notes Tropy + vault.
All three PDFs + Tropy + vault (the default).

Vault chunks come into the prompt with type : note and a TITRE field that's normally a short title — the note's frontmatter title, or its first H1 heading, or the bare filename without folder path or extension (extractTitle() in ObsidianMarkdownParser.ts). The full relative path (e.g. Reading notes/Hobsbawm-1962.md) is only used as a last-resort fallback when none of those exist (retrieval-service.ts: h.note.title || h.note.relativePath). Clicking a note citation in the assistant's reply opens it via the configured sources:open-note handler (typically the system's default markdown app — Obsidian itself, if it's registered).

Using the vault from external MCP clients

If you've enabled ClioDeck's MCP server (see the MCP Integration Guide), the vault is reachable from Claude Desktop / Claude Code as the search_obsidian tool:

search_obsidian(query: string, topK?: int = 10)

The search is BM25 over FTS5, returning the top-K matching notes with a short snippet. It does not use dense embeddings — that's only for in-app Brainstorm retrieval.

What's indexed, what isn't

The parser handles:

  • Standard markdown body.
  • YAML frontmatter (used for metadata; not embedded as content).
  • [[wikilinks]] and #tags (preserved in the text; useful as anchors).
  • ![[embeds]] (the link itself is preserved; the embed is not followed).

The parser ignores:

  • Files outside the vault root.
  • The .obsidian/ directory.
  • Non-.md files (images, PDFs inside the vault, etc.). For PDFs you keep in your vault, add them to ClioDeck's bibliography to index them as secondary sources.
  • Notes skipped for a structural reason: hidden file, empty note, binary content, a frontmatter parse error, oversized, excluded by an ignore pattern, unreadable, or a note whose only content is broken wikilinks (SkipReason in backend/integrations/obsidian/scan-report.ts). There is no user-settable frontmatter flag to opt a note out. There is also no per-note skip reason surfaced anywhere today — the indexer has no logging and the IPC handler only returns aggregate indexed/skipped/failed counts, not which note was skipped or why.

Storage and migration

The vault index used to live in its own .cliodeck/obsidian-vectors.db file. A later consolidation (obsidianStorePath() in ObsidianVaultIndexer.ts) folded it into the same shared .cliodeck/brain.db that PDF and Tropy vectors already used, under obsidian_*-prefixed tables (obsidian_notes, obsidian_chunks, obsidian_chunks_fts) to avoid name collisions. Opening a project that still has the legacy standalone file triggers an automatic one-time migration (rename if brain.db doesn't exist yet, or ATTACH + INSERT OR IGNORE if it does) — you should never need to do this by hand. On project load, ClioDeck also auto-migrates older nested layouts (.cliodeck/v2/*) to the flat scheme via migrateWorkspaceToFlat, independently of this.

The index itself is purely derived data and safe to lose — but because it now lives in the shared brain.db, there is no longer a way to delete just the Obsidian index by removing a file; doing so removes your PDF and Tropy indexes and research-journal history too. Use Settings → Obsidian Vault → Reindex to rebuild the vault's own tables in place instead (see the Unlink warning above for the one UI path that still does a raw file delete).

⚠️ Real risk, verified against code (not yet reproduced as a crash): four independent classes each open their own better-sqlite3 connection to the shared brain.db — VectorStore.ts (PDFs), PrimarySourcesVectorStore.ts (Tropy), HistoryManager.ts (research journal), and this page's own ObsidianVaultStore.ts. Only the Obsidian store sets journal_mode = WAL; none of the four — including Obsidian's — sets a busy_timeout. Compare with the separate journal.db (AI usage journal), where the developer explicitly added both WAL and busy_timeout = 3000 with a comment reasoning about exactly this: "the app and CLI can write at the same time — wait for the lock rather than fail with SQLITE_BUSY." The same reasoning was never applied to brain.db, despite it having more concurrent writers, not fewer. In practice: an Obsidian reindex running at the same moment as a Tropy watcher-triggered auto-resync (see Tropy guide) or a PDF batch-index could have one of the two throw an uncaught SQLITE_BUSY: database is locked and fail, even though they write to completely different tables in the same file — better-sqlite3's default busy timeout is 0ms (fail immediately, don't wait). Filed as issue #29.

See also

Clone this wiki locally