Skip to content

green-dalii/obsidian-llm-wiki

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

644 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

llm_wiki_banner

๐Ÿง  Karpathy LLM Wiki Plugin for Obsidian

AI-powered structured knowledge base that ingests your notes and generates a connected Wiki โ€” based on Andrej Karpathy's LLM Wiki concept.

Obsidian official score 95/100 | Native support for 10 languages | Zero-embedding graph retrieval | Full data sovereignty | Works with every provider

Version License Obsidian Compatibility Languages Providers
Maintenance Build Status Author
GitHub Stars Downloads Release Obsidian plugin Ask DeepWiki

English | ็ฎ€ไฝ“ไธญๆ–‡ | ็น้ซ”ไธญๆ–‡ | ๆ—ฅๆœฌ่ชž | ํ•œ๊ตญ์–ด | Deutsch | Franรงais | Espaรฑol | Portuguรชs | Italiano

Official Site | Obsidian Marketplace | Blog | Feedback & Discussion

๐Ÿš€ Quick Start | โœจ Features | ๐Ÿค– Model Selection Guide | ๐Ÿ”’ Transparency & Compliance | โ“ FAQ

ko-fi โ† If this plugin has helped you, feel free to buy me a coffeeโ™ฅ๏ธ or drop a star๐ŸŒŸโ†—


โšก Quick Update Reminder: This project evolves rapidly with frequent bug fixes, performance improvements, new features, and UX optimizations. We recommend updating to the latest version regularly in Obsidian (Settings โ†’ Community plugins โ†’ Check for updates), or enabling automatic plugin updates to ensure the best experience.

๐Ÿ“‘ Contents


๐Ÿ’ก What is LLM-Wiki?

You write. AI organizes. You ask. That's it.

๐ŸŽฏ The problem. Your notes are a goldmine โ€” people, concepts, ideas, connections. But right now they're just files in folders. Finding what relates to what means searching, tagging, and hoping you remember the thread.

โœจ The fix. Andrej Karpathy suggested something elegant: treat your notes as raw material, and let an LLM do the architect work. It reads what you write, pulls out entities and concepts, and weaves them into a structured Wiki โ€” complete with [[bidirectional links]], an auto-generated index, and a chat interface that answers questions from your knowledge.

๐Ÿ“š So you don't have to be the librarian. No deciding what deserves a page. No maintaining cross-links. No wondering if something is out of date. Pick any note (or folder, or selection) from your vault โ€” the LLM reads, extracts, writes, links, and even flags contradictions โ€” while you stay in flow.

๐Ÿค– And it's not another chatbot. ChatGPT knows the internet. LLM-Wiki knows you โ€” or rather, what you've taught it. Every answer carries [[wiki-links]] back into your knowledge graph. Every response is a trailhead, not a dead end.

๐Ÿ† Key differentiator โ€” Graph-powered retrieval at zero embedding cost. Most knowledge-base plugins use vector embeddings (expensive, per-provider, internet-dependent). LLM-Wiki runs Personalized PageRank over your existing [[wiki-link]] graph โ€” matching embedding-grade retrieval quality with zero API calls, no new dependencies, and full local-model support. Add zero-LLM Tier B section extraction (i18n-aware, 10 languages) and you get a knowledge engine that works for every user, on every provider.


โšก Why Obsidian + LLM-Wiki?

Obsidian is brilliant at linked thinking. But there's a catch: you're the one doing all the linking.

LLM-Wiki flips that. Instead of you building the graph by hand, the AI grows it with you. Add a note about a new concept โ€” it finds the connections you'd miss. Ask a question โ€” it walks your own knowledge graph and brings back answers with citations.

  • ๐Ÿ”— Your Graph View comes alive. New notes don't just sit there โ€” they sprout links to entities, concepts, and sources. The graph grows organically, and the plugin maintains it: detecting duplicates, fixing dead links, bridging languages with aliases.
  • ๐Ÿ’ฌ Your notes learn to talk back. Search becomes conversation. "What did I write about X?" becomes a dialogue, with streaming responses and [[wiki-links]] as breadcrumbs. Every answer is a path deeper into your own knowledge.
  • ๐Ÿง  Obsidian becomes a thinking partner. It stops being a cabinet for notes and starts being something that helps you think โ€” surfacing hidden connections, flagging contradictions, remembering what you forgot you knew.

๐Ÿš€ Quick Start

๐Ÿ“ฆ Installation

๐ŸŒŸ Recommended โ€” Obsidian Community Plugin Market:

  1. In Obsidian, go to Settings โ†’ Community plugins
  2. Click Browse and search for "Karpathy LLM Wiki"
  3. Click Install, then Enable

๐ŸŒ Or from the Community Plugin website โ€” visit community.obsidian.md/plugins/karpathywiki and click Add to Obsidian to install directly.

โš™๏ธ Manual (alternative):

  1. Download main.js, manifest.json, styles.css from Releases
  2. In Obsidian, go to Settings โ†’ Community plugins. On the Installed plugins tab, click the folder icon to open your plugins directory
  3. Create a folder named karpathywiki, drop the three files inside
  4. Back in Obsidian, click the refresh icon โ€” Karpathy LLM Wiki will appear under Installed plugins
  5. Toggle it on to enable

๐Ÿ”จ Development: git clone, pnpm install, pnpm build.

๐Ÿ”„ Updating

This project evolves rapidly โ€” new features, bug fixes, and improvements are shipped frequently. We recommend keeping up to date:

Option A โ€” Manual update (recommended):

  1. Go to Settings โ†’ Community plugins
  2. Click Check for updates
  3. Find Karpathy LLM Wiki in the list and click Update

Option B โ€” Enable auto-update:

  1. Go to Settings โ†’ Community plugins
  2. Toggle on Automatically check for plugins updates
  3. New versions will be detected automatically; update manually at your convenience

๐Ÿ’ก Why stay updated? Each release may include new features, performance improvements, and important bug fixes. We actively maintain this plugin โ€” missing updates means missing out on a better experience.

๐Ÿ”‘ Configure an LLM Provider

  1. Open Settings โ†’ Karpathy LLM Wiki
  2. Pick a provider from the dropdown (Anthropic, Anthropic Compatible, Google Gemini, OpenAI, DeepSeek, Kimi, GLM, MiniMax, LM Studio, Ollama, OpenRouter, or custom)
  3. Enter your API key (not needed for Ollama)
  4. Click Fetch Models to populate the model dropdown, or type a model name manually
  5. Click Test Connection, then Save Settings

๐Ÿฆ™ Ollama (local, no API key): Install Ollama, pull a model (ollama pull gemma4 or ollama pull qwen3.5:27b), select "Ollama (Local)" in the provider dropdown.

๐ŸŽ›๏ธ LM Studio (local, no API key): Install LM Studio, start its local server (default http://localhost:1234/v1), select "LM Studio (Local)" in the provider dropdown. LM Studio runs a built-in OpenAI-compatible server โ€” API key field is optional.

See Model Selection Guide for details.

๐ŸŽฎ Usage

Method How
๐Ÿ“ฅ Ingest single source Cmd+P โ†’ "Ingest single source" โ€” select a note (Markdown or PDF, v1.25.0+) to extract entities and concepts into Wiki pages
๐Ÿ“‚ Ingest from folder Cmd+P โ†’ "Ingest from folder" โ€” pick a folder, batch generate Wiki from all notes inside
๐Ÿ“‘ Ingest multiple files Cmd+P โ†’ "Ingest multiple files" โ€” pick specific notes via two-pane modal (recursive folder tree + per-file checkboxes), then batch ingest the selection (with live queue + per-file cancel)
๐ŸŽฏ Ingest current file Click the sticker icon in the left ribbon, or Cmd+P โ†’ "Ingest current file" โ€” ingest the file you're editing
๐Ÿ” Query wiki Cmd+P โ†’ "Query wiki" โ€” conversational Q&A over your Wiki, streaming responses with [[wiki-links]]
๐Ÿ› ๏ธ Lint wiki Cmd+P โ†’ "Lint wiki" โ€” full health scan: duplicates, dead links, empty pages, orphans, missing aliases, contradictions. Schema suggestions surfaced inside the Lint Modal
๐Ÿ“‹ Regenerate index Cmd+P โ†’ "Regenerate index" โ€” rebuild wiki/index.md with current pages and aliases
๐Ÿ“Š View Ingestion History (v1.21.0) Cmd+P โ†’ "View Ingestion History" โ€” browse past ingestions, lint reports, and maintenance runs in a searchable, filterable UI
โน Cancel current operation Cmd+P โ†’ "Cancel current ingestion" โ€” stop an in-progress operation cleanly at the next batch boundary
๐ŸŽ‰ Recreate Welcome Note (v1.23.0) Cmd+P โ†’ "Recreate Wiki Welcome Note" โ€” re-generate the first-run Welcome note

Re-ingesting the same source does incremental updates on entity/concept pages (new info merged in). Summary pages are regenerated.

๐Ÿ’ก Smart Batch Skip: When ingesting a folder, the plugin automatically detects already-processed files and skips them to save time and API costs. The batch report shows skipped count.

Command palette โ€” search "karpa" to see all Karpathy LLM Wiki commands

โš ๏ธ Upgrading from an Older Version?

๐Ÿ”ง Upgrading from v1.24.x. PDF ingest (v1.25.0) writes its cache to .obsidian/plugins/karpathywiki/pdf-cache/ (up to 100 MB / 1000 entries / 10 MB single-entry; LRU-by-mtime eviction on startup and at each batch ingest). Your vault is not modified by default โ€” turn on Write PDF Markdown to Vault (Settings โ†’ Wiki Configuration โ†’ Wiki Folder) only if you want a <basename>.pdf.md sidecar next to the source PDF. Two new settings โ€” Force PDF Support (advanced, default off) and Write PDF Markdown to Vault (default off) โ€” are fully backward-compatible: old data.json without them defaults to false.

๐Ÿ”ง Upgrading from v1.24.0. The internal <!-- reviewed: keep --> comment marker (v1.24.0, #244) that protected only a page's Mentions in Source section has been removed. To keep a curated Mentions section, set reviewed: true in the page frontmatter โ€” it protects the whole page (Mentions included) and, unlike the hidden comment, stays visible in the Properties panel and survives Markdown linters.

Backward compatible. No breaking changes since v1.0.0 โ€” your existing wiki pages, settings, and workflows are preserved without reconfiguration.

After upgrading, run Lint Wiki โ†’ Smart Fix All for a one-click, causality-ordered repair:

  1. ๐Ÿท๏ธ Complete Aliases (LLM batch-generates translations, acronyms, alternate names)
  2. ๐Ÿ”„ Merge Duplicates (cross-language, abbreviation, and high-similarity matches)
  3. ๐Ÿ”— Fix Dead Links / Link Orphans / Expand Empty Pages

Then Regenerate Index to rebuild wiki/index.md with alias entries for every page, enabling alias-aware search (e.g., "DSA" finds "DeepSeek-Sparse-Attention").

๐Ÿ“– Detailed upgrade walkthroughs for specific version jumps (v1.20.3 slug fingerprint, v1.16.0 double-nested links) are maintained in GitHub Discussions / Upgrading.

Settings to review: Force PDF Support (Settings โ†’ LLM Configuration โ†’ Advanced, off by default โ€” only needed for non-NATIVE providers), Write PDF Markdown to Vault (Settings โ†’ Wiki Configuration โ†’ Wiki Folder, off by default), Wiki Output Language (independent from UI), Extraction Granularity (Minimalโ€“Fine, + Custom), Page Generation Concurrency (default 3), Batch Delay (default 300ms).

โšก What's New in v1.25.0

Four themes: cache-only PDF ingest, local-model guidance, prompt centralization for PDF transcriber, and eight e2e bug fixes. Recommended upgrade for everyone on v1.24.x.

  • ๐Ÿ“„ PDF Ingest (Level 1). Pick a PDF from your vault โ€” the plugin reads it through your LLM provider's native file input (anthropic / openai / bedrock-anthropic / bedrock-openai; any other OpenAI/Anthropic-compatible endpoint with Force PDF Support enabled in Settings โ†’ LLM Configuration โ†’ Advanced), converts it to Markdown via OCR-style verbatim transcription, and re-enters the regular Markdown ingest pipeline. Every existing entity/concept/alias/[[wiki-link]] workflow applies unchanged. Result is content-hash cached in .obsidian/plugins/karpathywiki/pdf-cache/ (key includes converterVersion so prompt upgrades invalidate stale entries). See the Local PDF OCR Path for the recommended local-Apple-Silicon setup.
  • ๐Ÿ—„๏ธ Bounded cache growth. Three-defense-layer cache housekeeping (100 MB total / 1000 entries / 10 MB single-entry caps) with LRU-by-mtime eviction. Old entries pruned on startup and at the start of every batch ingest. Cache-only by default โ€” your vault is not modified.
  • ๐Ÿ“ Optional vault sidecar (advanced). Settings โ†’ Wiki Configuration โ†’ Wiki Folder โ†’ Write PDF Markdown to Vault writes <basename>.pdf.md next to the source PDF after conversion. Off by default.
  • ๐Ÿฆ™ Local model recommendations. The Model Selection Guide now has dedicated Local and Cloud sections covering Qwen3.5 / Qwen3.6 / Gemma 4 (parameter-vs-quality tradeoffs, MLX vs GGUF quantization, context strategy).
  • ๐Ÿ›ก๏ธ Verbatim PDF transcriber prompt. The PDFโ†’Markdown prompt is re-framed as OCR-style verbatim conversion with [illegible] / [figure: ...] / [equation: ...] anti-hallucination markers; small/local models that wrap output in ```markdown fences are auto-cleaned before cache write. Prompt centralized into src/wiki/prompts/pdf.ts alongside every other LLM-call prompt in the project.
  • โน Cancelable PDF ingest. Clicking the status bar mid-conversion aborts the in-flight LLM call through Vercel AI SDK v6 AbortSignal within ~200 ms.
  • ๐ŸŒ i18n completeness โ€” 10 new keys per locale for the two new settings + PDF Ingest + Local PDF OCR Path (force-pdf-support toggle, write-pdf-markdown-to-vault toggle, source-rejected-pdf-unsupported Notice).

Settings to review: Force PDF Support (Settings โ†’ LLM Configuration โ†’ Advanced, default off โ€” only relevant for non-NATIVE providers), Write PDF Markdown to Vault (Settings โ†’ Wiki Configuration โ†’ Wiki Folder, default off โ€” opt-in sidecar).

โœจ Features

๐Ÿ“Š Knowledge Quality

  • ๐Ÿ” Entity/Concept extraction โ€” LLM extracts entities (people, organizations, products, events, etc.) and concepts (theories, methods, terms, etc.) from your notes and generates standalone Wiki pages. Flexible extraction granularity (minimal ~5, coarse ~10, standard ~50, fine ~100, custom 1โ€“500) balances analysis depth with API cost.
  • ๐Ÿท๏ธ Mandatory page aliases โ€” every generated page includes at least 1 alias (translation, abbreviation, variant) for cross-language duplicate detection.
  • ๐Ÿ”„ Duplicate detection & merge โ€” semantic tiered detection catches true duplicates (cross-language translations, abbreviations, spelling variants); smart LLM fusion merges content while preserving aliases.
  • ๐Ÿงฉ Intelligent knowledge fusion โ€” multi-source updates merge new information without duplication; contradictions are preserved with source attribution; reviewed: true pages are protected from overwrite.
  • ๐Ÿ“ Content truncation guard โ€” 8000 max_tokens with automatic stop_reason detection and 2ร— token retry, covering all providers.
  • ๐Ÿ“ Original quote preservation โ€” Mentions-in-Source sections preserve quotes in their original language (optional translation) for full traceability.
  • ๐ŸŽจ Customizable tag vocabulary (v1.18.0). Settings โ†’ Wiki โ†’ Tag Vocabulary Mode โ†’ Custom lets you define your own entity-type and concept-type tag lists (e.g. Medical_Arzneimittel, ๆณ•่ง„). The plugin respects your vocabulary in extraction prompts and frontmatter validation; the existing Lint audit (Issue #85 v7) reports any page whose tags fall outside the active vocabulary.

Custom tag vocabulary chip inputs

๐Ÿ“„ PDF Ingest (v1.25.0)

Pick a PDF from your vault โ€” the plugin reads it through your LLM provider's native file input, converts it to Markdown, and re-enters the regular Markdown ingest pipeline. Every existing entity/concept/alias/[[wiki-link]] workflow applies unchanged.

  • ๐Ÿ”Œ Provider gate โ€” Anthropic, OpenAI, Bedrock Anthropic, and Bedrock OpenAI handle PDF natively. For any other OpenAI/Anthropic-compatible endpoint, enable Force PDF Support in Settings โ†’ LLM Configuration โ†’ Advanced to let the plugin attempt the call (your endpoint decides; failures surface as a localized Notice guiding you to disable the toggle). See the Local PDF OCR Path below for the recommended local setup.
  • ๐Ÿ—„๏ธ Content-hash cache โ€” identical PDF + model + converter version returns the cached Markdown with no LLM call. The cache lives in .obsidian/plugins/karpathywiki/pdf-cache/; cache key embeds a converterVersion so prompt upgrades invalidate stale entries automatically.
  • ๐Ÿ“ Bounded growth โ€” three-defense-layer cache housekeeping (100 MB total / 1000 entries / 10 MB single-entry caps) with LRU-by-mtime eviction; old entries are pruned on startup and at the start of every batch ingest. Cache only โ€” your vault is not modified by default.
  • ๐Ÿ“ Optional vault sidecar โ€” Settings โ†’ Wiki Configuration โ†’ Wiki Folder โ†’ Write PDF Markdown to Vault writes <basename>.pdf.md next to the source PDF after conversion. Off by default (cache-only).
  • ๐Ÿ›ก๏ธ Verbatim transcriber prompt โ€” the PDFโ†’Markdown prompt is re-framed as OCR-style verbatim conversion with [illegible] / [figure: ...] / [equation: ...] anti-hallucination markers; small/local models that wrap output in ```markdown fences are auto-cleaned before cache write.
  • โน Cancellable โ€” clicking the status bar mid-conversion aborts the in-flight LLM call through Vercel AI SDK v6.

๐Ÿ’ฌ Query & Feedback

  • ๐Ÿ” 5-stage PPR seed-selection cascade (v1.24.1 PATCH). When you ask a multi-hop question, Query Wiki composes the answer through five complementary stages before any generation runs:

    1. Lex fast path โ€” straight token-overlap against every entity/concept title and aliases (free, instant; gates the rest).
    2. LLM keyword generation โ€” the LLM proposes 8โ€“12 cross-language keywords from your query (handles synonyms, abbreviations, token-overlap-resistant terms).
    3. Local substring scan โ€” every generated keyword is re-matched locally across page titles, aliases, and body snippets (no extra LLM call; rounds out noise-tolerant recall).
    4. LLM KB fallback โ€” when lex + keyword scan returns weak signals, the LLM re-seeds the top-N candidates against the full wiki for one semantic pass.
    5. PPR graph expansion โ€” Personalized PageRank (Haveliwala 2002) over the [[wiki-link]] graph starting from the candidate seed set; gives the LLM-graph-aware multi-hop context that linear search can't reach.

    The cascade automatically truncates to whichever step returned enough signal โ€” no fixed 5-step cost, no LLM calls when lex is sufficient, no lost precision when the LLM augmentation is needed. End-to-end relevance (PPR @5 = 27.1% on the project's own benchmark corpus) outperforms pure-knn baselines (24.1%) without opt-in embedding. Stage 1.5 (steps 2โ€“3) handles the multi-hop query types that pure lex misses; Stage 1.7 (step 4) recovers when the LLM-injected keywords haven't seen enough signal yet; Stage 1.9 (step 5) guarantees that the LLM sees neighbor context, not just a flat top-N list. Replaces the older binary-tier cascade.

  • ๐Ÿค– Conversational query โ€” ChatGPT-style dialog with streaming Markdown output, automatic [[wiki-links]], and multi-turn history.

  • ๐ŸชŸ Right-docked side panel (v1.22.1, PR #196). Query Wiki opens in a Copilot-style right sidebar leaf (reusing an existing leaf if already open) instead of a centered popup. The message-circle ribbon icon and Query Wiki command activate/reveal the panel; your notes stay visible alongside the conversation. All functionality is preserved unchanged.

Query Wiki side panel

  • ๐Ÿ“ค Query โ†’ Wiki feedback โ€” save valuable conversations back into the Wiki, with entity/concept extraction and pre-save semantic dedup.
  • ๐Ÿ”’ Duplicate-save guard โ€” hash tracking prevents unchanged conversations from re-evaluating.

๐Ÿ› ๏ธ Maintenance

  • ๐Ÿ” Lint health scan โ€” single comprehensive report detects: duplicate pages, dead links, empty pages, orphans, missing aliases, contradictions.
  • ๐ŸŽฏ Tiered semantic duplicate detection โ€” Tier 1 (direct name match: cross-language, abbreviation, high-similarity titles) always verified; Tier 2 (indirect signals: shared links, medium similarity) fills token budget.
  • โšก One-click Smart Fix All โ€” batch fixes in causal order: fill aliases โ†’ merge duplicates โ†’ fix dead links โ†’ link orphans โ†’ expand empty pages, with per-phase popup report.
  • ๐Ÿท๏ธ Alias completion โ€” one-click parallel batch generation of missing aliases, improving future duplicate detection.
  • ๐Ÿ”„ Auto-maintenance โ€” multi-folder watching, scheduled Lint, startup health check (all optional).
  • โš ๏ธ Contradiction state machine โ€” detected โ†’ review-passed โ†’ resolved (AI-fix) or detected โ†’ unresolved (manual).
  • ๐Ÿ›ก๏ธ Pre-ingest requirements gate (v1.21.0) โ€” every source file is validated before any LLM call: empty/whitespace/frontmatter-only notes are rejected, and content-hash dedup catches identical files across paths. Prevents small/local models from hallucinating entity names on blank inputs.
  • ๐Ÿ“Š Operation History Panel (v1.21.0) โ€” searchable, filterable UI for past ingestions, lint reports, and maintenance runs, with insight-driven KPI cards and clickable page links.

History Panel

  • ๐Ÿงน Incomplete-page cleaner (v1.21.0) โ€” pages left in a partial state after interrupted ingests are automatically archived on startup. Recoverable from Obsidian's .trash.

๐ŸŒ LLM & Language

  • ๐Ÿ”Œ Multi-provider support โ€” Anthropic, Anthropic-compatible (Coding Plan), Gemini, OpenAI, DeepSeek, Kimi, GLM, MiniMax, LM Studio, OpenRouter, Ollama, custom endpoint.
  • ๐Ÿ”„ 5xx auto-retry โ€” exponential backoff on HTTP 5xx / 429 / 529 across all clients (max 2 retries).
  • ๐Ÿ“‹ Dynamic model list โ€” fetched live from the provider API.
  • ๐ŸŒ Wiki output language โ€” 10 languages independent of UI (English / ็ฎ€ไฝ“ไธญๆ–‡ / ็น้ซ”ไธญๆ–‡ / ๆ—ฅๆœฌ่ชž / ํ•œ๊ตญ์–ด / Deutsch / Franรงais / Espaรฑol / Portuguรชs / Italiano), with custom input option.
  • ๐ŸŒ Full UI internationalization โ€” plugin UI in 10 languages with 269+ UI fields fully translated to natural local expression.
  • โšก Rate-limit guardian โ€” automatically detects when parallel generation triggers rate limits and prompts to lower concurrency, increase batch delay, or switch provider.
  • ๐Ÿฆ™ Web Clipper compatibility โ€” one-click addition of the official Obsidian Web Clipper's Clippings/ folder to the watch list; clipped web pages auto-ingest into the Wiki.

๐Ÿ—๏ธ Architecture & Performance

  • ๐Ÿ•ธ๏ธ PPR over [[wiki-link]] graph (v1.24.0+, mature in v1.24.1 PATCH). Personalized PageRank (Haveliwala 2002) runs over the directed graph of [[wiki-link]] edges between your wiki pages; the cascade seeds PPR at the cascade's top-N candidate set so multi-hop context travels through up to 3 expansion rings. This is what makes the Query Wiki answers graph-aware (a "Founders of Microsoft" question resolves via Bill Gates โ†’ Microsoft โ†’ competitors, not just literal title overlap). 2,137-page vaults typically see <100 ms warm + 3-hop expansions, regardless of vault size. Used by all 4 stages of the seed-selection cascade (Query & Feedback section above) and by lint duplicate detection when indirect links tie two candidate pages together.
  • โšก Parallel page generation โ€” configurable 1โ€“5 concurrent pages, default 3 (parallel), 2โ€“3ร— speedup on large sources; per-page error isolation.
  • ๐Ÿ“š Iterative batched extraction โ€” adaptive batch sizing eliminates the long-document max_tokens bottleneck.
  • ๐Ÿ›๏ธ Three-layer architecture โ€” Your vault notes (read-only) โ†’ wiki/ (LLM-generated pages organized as wiki/sources/, wiki/entities/, wiki/concepts/) โ†’ schema/ (co-evolved configuration).
  • ๐Ÿงฉ Modular codebase โ€” 20+ focused modules in src/.

๐Ÿ”’ Privacy & Security

  • No backend, no tracking. The plugin runs entirely inside Obsidian โ€” no external servers, no analytics, no data collection of any kind. Unless you actively configure an LLM provider, your notes never leave your vault.
  • Data stays local by default. The plugin does not store, cache, or transmit your content anywhere outside of the LLM API you have chosen. Only the text you send for ingestion or query leaves your device โ€” and only to the provider you configured.
  • Full local mode via Ollama, LM Studio, or local providers. For complete data sovereignty, use a locally running LLM. Your notes are processed entirely on your machine โ€” never touching the internet.
  • Minimal permissions. Vault file access is used for Wiki management (reading notes, generating pages, detecting dead links). Network access is used only for communicating with your chosen LLM provider's API. Clipboard access is limited to the "Copy" button in the Query modal โ€” used only when you click it.

๐Ÿ“– Example

Input: sources/machine-learning.md

### Machine Learning
Machine learning uses algorithms to learn from data.

### Types
- Supervised learning
- Unsupervised learning
- Reinforcement learning

Output โ€” Concept page: wiki/concepts/supervised-learning.md

---
type: concept
created: 2025-12-01
updated: 2026-05-15
sources: ["[[sources/machine-learning]]"]
tags: [method]
aliases: ["็›‘็ฃๅญฆไน ", "Supervised Learning"]
---

### Supervised Learning

### Basic Information
- Type: method
- Source: [[sources/machine-learning]]

### Description
Supervised learning is a machine learning paradigm where models learn
from labeled training data to make predictions on unseen data...

### Related Concepts
- [[concepts/Machine-Learning|Machine Learning]]
- [[concepts/Unsupervised-Learning|Unsupervised Learning]]

### Related Entities
- [[entities/Arthur-Samuel|Arthur Samuel]]

### Mentions in Source
- "Supervised learning uses labeled data to train predictive models..."

๐Ÿค– Model Selection Guide

This plugin follows Karpathy's philosophy: feed the LLM full Wiki context, not chunked RAG retrieval. Long-context models are strongly recommended โ€” the larger your Wiki grows, the more context the LLM needs.

๐Ÿ’ก Why not RAG? Karpathy's original critique argues that RAG fragments knowledge and breaks the LLM's ability to reason across the full knowledge graph.

๐Ÿ’ฐ Value-First Strategy: You don't need flagship models. The following cost-effective alternatives deliver excellent results at lower prices:

โ˜๏ธ Cloud Model Picks

Tier Model Context Why
๐ŸŒŸ Value Pick DeepSeek V4-Flash 1M tokens Lowest cost ($0.14/M), 284B MoE, ideal for batch ingestion
๐ŸŒŸ Value Pick Gemini-3.5-Flash 1M tokens 4ร— faster output than GPT-5.5, great for agent tasks
๐ŸŒŸ Value Pick Qwen3.6-Plus 1M tokens Strong coding & agentic capabilities, competitive pricing
๐ŸŒŸ Value Pick Grok-4 2M tokens xAI 2025-07 flagship, 2M context, strong reasoning & code tasks
Balanced Claude Sonnet 4.6 1M tokens Great quality/cost balance, $3/$15 per million tokens
Lightweight Claude Haiku 4.5 200K tokens Fast and affordable for smaller wikis
Budget Xiaomi MiMo-V2.5 1M tokens Xiaomi 310B/15B MoE, MIT open-source 2026-04, agent & multimodal
Flagship Claude Opus 4.7 1M tokens Ultimate quality, higher cost โ€” use selectively
Flagship GPT-5.5 1M tokens Top reasoning, higher cost โ€” use selectively

๐Ÿฆ™ Local Model Recommendations (Ollama / LM Studio)

Local inference wins on data sovereignty, offline use, and zero API cost. The trade-off is context window (most sit between 8Kโ€“128K; recent open-weight families reach 262K) and instruction following vs. flagship cloud models. Pick by your hardware budget: larger parameter counts buy world knowledge and instruction fidelity (better extraction quality, fewer hallucinations); smaller counts buy speed and memory headroom but pay for it in hallucinations and weaker long-context reasoning. The sweet spot on a 24 GB Apple Silicon or single consumer GPU is the 27Bโ€“35B-A3B class.

Model Params Context Why
Qwen3.5 27B 27B dense 262K Best quality/size trade-off for ingest; MLX 4-bit fits in 24 GB
Qwen3.5 35B-A3B 35B total / 3B active MoE 262K Faster than 27B dense at similar quality; ideal memory saver
Qwen3.5 122B-A10B 122B / 10B MoE 262K Quality ceiling; needs โ‰ฅ48 GB VRAM or dual GPUs
Qwen3.6 27B 27B dense 256K+ 2026-04 refresh over Qwen3.5 27B โ€” prefer this if your hardware can take it
Qwen3.6 35B-A3B 35B / 3B MoE 262K Same trade-off as Qwen3.5 35B-A3B, newer weights
Gemma 4 31B IT 31B dense 262K Strong instruction following, clean Markdown output
Gemma 4 26B A4B IT 26B / 4B MoE 262K Lower memory than 31B dense at comparable quality
Gemma 4 E2B / E4B IT 2B / 4B 131K CPU-runnable; use only for small wikis or quick previews

Quantization: MLX 4-bit on Apple Silicon is typically 1.5โ€“2ร— faster than GGUF Q4_K_M at the same effective bitrate. GGUF Q4_K_M is the default cross-platform choice; reach for Q5/Q8 only if you have the VRAM headroom and notice quality regression at Q4.

Context strategy: When your Wiki grows past ~500 pages, a 262K local model still sees most of the context the Query engine assembles, but ingest on a 2000-page vault will outrun it. Common pattern: cloud for ingest + local for query. For full local setups, the 27B/35B-A3B class is the sweet spot.

๐Ÿ“„ Local PDF OCR Path (v1.25.0+)

v1.25.0's PDF ingest works with any provider that accepts PDF as a file part. For a fully local pipeline on Apple Silicon (the only platform oMLX currently supports), the recommended setup is:

  1. Install oMLX and enable its built-in Markitdown backend (local PDFโ†’Markdown conversion).
  2. Load Baidu Unlimited-OCR (open-sourced 2026-06-22, 3B total / 0.5B active, end-to-end OCR that handles long documents without the "slower the longer it generates" failure mode of older OCR models) as the vision model in oMLX.
  3. In this plugin: set provider to Custom OpenAI-Compatible (oMLX speaks the OpenAI-compatible protocol), point the Base URL at oMLX's local server, turn on Force PDF Support in Settings โ†’ LLM Configuration โ†’ Advanced, and pick the multimodal model oMLX is serving for the ingest summarization.

The PDF never leaves your machine โ€” Markitdown does the structural conversion locally, Unlimited-OCR does the visual recognition locally, and the local LLM does the summarization locally. The plugin's cache (.obsidian/plugins/karpathywiki/pdf-cache/) then keeps re-ingests instant.

Fallback: if oMLX/Markitdown is unavailable (Linux/Windows or older Macs), point Force PDF Support directly at a local multimodal LLM that accepts PDF file parts โ€” quality is good when the model is large enough, but VRAM demands scale steeply with page count.

๐Ÿ”Œ Anthropic Compatible (Coding Plan): If your provider offers an Anthropic-compatible API endpoint, select "Anthropic Compatible" and enter your provider's Base URL and API Key.

๐Ÿ’ก Subscription plans: Coding Plan, OpenAI Pro, or Anthropic Pro plans are excellent options for cost control with frequent use. This plugin supports these services.


๐Ÿ—๏ธ Architecture

Karpathy's three-layer separation design:

๐Ÿ“„ Your vault notes (any folder)   # ๐Ÿ“– You pick which notes to ingest
  โ†“ ingest
wiki/                              # ๐Ÿง  LLM-generated Wiki pages (wiki/sources/, wiki/entities/, wiki/concepts/)
  โ†“ query / maintain
schema/                            # ๐Ÿ“‹ Wiki structure configuration (naming, templates, categories)

๐Ÿ“– See the full codebase structure in CONTRIBUTING.md โ†’ Project Structure.

Generated pages:

  • wiki/sources/filename.md โ€” ๐Ÿ“„ Source summary
  • wiki/entities/entity-name.md โ€” ๐Ÿ‘ค Entity pages (people, orgs, projects, etc.)
  • wiki/concepts/concept-name.md โ€” ๐Ÿ’ก Concept pages (theories, methods, terms, etc.)
  • wiki/index.md โ€” ๐Ÿ“‘ Auto-generated index
  • wiki/log.md โ€” ๐Ÿ“ Operation log

โ“ FAQ

Keep your plugin updated โ€” new features and fixes ship frequently. Run Settings โ†’ Community Plugins โ†’ Check for updates regularly.

๐Ÿ“– More FAQs on GitHub Discussions.

What does the plugin actually do? Pick any note, folder, or selection from your vault; the LLM extracts entities and concepts and generates an interlinked wiki with [[bidirectional links]]. Ask questions and get conversational answers grounded in your notes โ€” not internet search. Generated summaries live under wiki/sources/, entities under wiki/entities/, and concepts under wiki/concepts/ โ€” your original vault notes are never modified.

Is my data sent to third parties? ๐Ÿ”’ Privacy first. No backend, no tracking, no analytics โ€” the plugin runs entirely inside Obsidian. Only text you explicitly send for ingestion/query leaves your device, and only to the LLM provider you configure. For complete data locality, use a local provider (Ollama or LM Studio with no API key) โ€” your data never touches the internet.

How is this different from RAG chatbots? Unlike chunked RAG that fragments context, LLM-Wiki runs a Personalized PageRank engine over your existing [[wiki-link]] graph โ€” finding related pages via link structure, not vector embeddings. This means zero embedding cost, no new dependencies, and full support for local and offline models.

Which LLM should I use? Long-context models (โ‰ฅ200K tokens) work best. Budget-friendly picks: DeepSeek V4-Flash ($0.14/M), Gemini 3.5 Flash, Qwen3.6-Plus. Local models via Ollama/LM Studio work for query but have smaller context windows (8Kโ€“128K). See the Model Selection Guide for details.

How do I get started? Install from Obsidian Community Plugins โ†’ pick an LLM provider โ†’ Test Connection โ†’ run Ingest single source (or Ingest from folder) on any note in your vault โ†’ your first wiki pages appear within seconds. See Quick Start above.

How can I control API costs? Use Coarse or Minimal extraction granularity for batch ingestion (fewer LLM calls). Smart Batch Skip auto-detects already-ingested files. Auto-Maintenance is OFF by default (enable only if needed). Lint shows counts before running fixes โ€” nothing is charged without your approval.

Is my existing wiki safe? โœ… Backward compatible since v1.0.0. Set reviewed: true on any page to protect it from overwrite. The plugin never modifies your original vault notes โ€” only generates new pages inside the wiki/ folder.

Can I use the plugin in my language? ๐ŸŒ 10 languages for both UI and wiki output: English, ็ฎ€ไฝ“ไธญๆ–‡, ็น้ซ”ไธญๆ–‡, ๆ—ฅๆœฌ่ชž, ํ•œ๊ตญ์–ด, Deutsch, Franรงais, Espaรฑol, Portuguรชs, Italiano. UI and wiki language are independent โ€” your wiki can be in Chinese while the interface stays in English. Adding an 11th language is contributor-driven (follow the Italian PR #159 pattern).

What minimum setup is needed? Obsidian v1.11.0+ (desktop: Windows/macOS/Linux). An LLM provider API key (or Ollama/LM Studio for local, no API key needed). The plugin's llmReady guard requires a successful connection test before core features unlock โ€” this prevents silent failures from misconfigured providers.

How do I cancel a running operation? Click the status bar text (shows "Ingestingโ€ฆ click to cancel") or Cmd+P โ†’ "Cancel current ingestion". Stops cleanly at the next batch boundary, preserving all completed work.

Where do I get help?

  • GitHub Issues โ€” bug reports
  • GitHub Discussions โ€” questions, feature requests, upgrade help
  • Developer Console (Ctrl+Shift+I / Cmd+Option+I) โ€” copy logs with module-name prefixes for faster diagnosis

๐Ÿ”’ Transparency & Compliance

This plugin is listed on the Obsidian Community Plugin Market and undergoes automated review for security and permissions.

The plugin has no backend, no server infrastructure, and no data collection of any kind. It is purely local software running inside Obsidian. The plugin cannot and does not collect, store, or transmit your data to any server โ€” because no such server exists.

Network access is used only to communicate with the LLM provider you configure โ€” no other network calls are made. This is entirely under your control: you choose the provider, you enter the API key, you decide where your data goes.

File system access (vault enumeration) is required to build and maintain the wiki: reading your source notes, generating pages, scanning for dead links, and detecting duplicate pages. The plugin never modifies your source files โ€” only files under the wiki folder.

Clipboard access is used exclusively by the "Copy" button in the Query modal, and only when you click it.

If you prefer complete data locality, use a local LLM provider such as Ollama or LM Studio. With a local provider, your data never leaves your machine.

๐Ÿ’– Support the Project

If LLM-Wiki has become a meaningful part of your knowledge workflow, you can support its ongoing development:

Sponsorship is entirely optional. The plugin stays Apache-2.0-licensed and feature-complete regardless.

Sponsors

Thanks to the following people for supporting the project:

๐Ÿ“œ License

Apache License, Version 2.0 โ€” see LICENSE and NOTICE.

๐Ÿ™ Acknowledgments

Star History

Star History Chart

About

Karpathy's LLM Wiki implementation plugin for Obsidian - multi-page knowledge generation with entity/concept pages and conversational query.

Topics

Resources

License

Code of conduct

Contributing

Stars

325 stars

Watchers

2 watching

Forks

Sponsor this project

Packages

 
 
 

Contributors

Languages