1.25.0

🌟 Karpathy LLM Wiki v1.25.0

Highlights

v1.25.0 is a MINOR release shipping PDF Ingest (Level 1) as a first-class source format. Pick a PDF from your vault — the plugin reads it through your LLM provider's native file input, converts it to Markdown via an OCR-style verbatim transcriber, caches the result in .obsidian/, then re-enters the regular Markdown ingest pipeline. Every existing entity / concept / alias / [[wiki-link]] workflow applies unchanged.

PDFs are now a first-class format alongside Markdown in the source picker (no pdfIngestEnabled toggle). The cache is the single source of truth for the LLM-converted Markdown — your vault is not modified by default. 2182 tests passing across 165 files (+102 since v1.24.1). Fully backward compatible — old data.json from v1.24.x continues to work unchanged.

This release also closes the Obsidian build verification warning that affected earlier releases: package-lock.json URLs have been rewritten from registry.npmmirror.com → registry.npmjs.org so Obsidian CI's npm install produces a main.js hash that matches the release asset.

Added

• 📄 PDF Ingest (Level 1). Pick a PDF from your vault — the plugin reads it through your LLM provider's native file input (anthropic / openai / bedrock-anthropic / bedrock-openai natively; any other OpenAI/Anthropic-compatible endpoint via Force PDF Support), converts it to Markdown via an OCR-style verbatim transcriber with [illegible] / [figure: ...] / [equation: ...] anti-hallucination markers, and re-enters the regular Markdown ingest pipeline. The result is content-hash cached in .obsidian/plugins/karpathywiki/pdf-cache/; the cache key embeds converterVersion so prompt upgrades invalidate stale entries automatically.

• 🗄️ Bounded cache growth. Three-defense-layer cache housekeeping: single-entry cap (10 MB) pre-write, LRU-by-mtime eviction (100 MB total / 1000 entries) post-write, and prepareBatchIngest() (TTL purge + size enforce) wired into runBatchIngest(). Cache-only by default — your vault is not modified.

• 📝 Optional vault sidecar. Settings → Wiki Configuration → Wiki Folder → Write PDF Markdown to Vault writes a <basename>.pdf.md sidecar next to the source PDF after conversion. Off by default (cache-only). This is the only user-visible opt-in that touches the vault.

• 🦙 Local model recommendations. Dedicated ### 🦙 Local Model Recommendations (Ollama / LM Studio) H3 in the Model Selection Guide, covering Qwen3.5 (27B / 35B-A3B / 122B-A10B), Qwen3.6 (27B with 256K+ context / 35B-A3B), Gemma 4 (E2B / E4B / 26B-A4B / 31B), with parameter-vs-quality tradeoff guidance, MLX-vs-GGUF quantization notes, and a context-strategy block.

• 📄 Local PDF OCR path on Apple Silicon. Documented end-to-end recommended setup for fully-local PDF ingestion: oMLX + Markitdown backend + Baidu Unlimited-OCR (open-sourced 2026-06-22, 3B total / 0.5B active, end-to-end OCR that solves the "slower the longer it generates" failure mode of older OCR models on long documents). Provider: Custom OpenAI-Compatible pointing at oMLX's local server with Force PDF Support on. PDF never leaves the machine.

• ⏹ Cancellable PDF ingest. Clicking the status bar mid-conversion aborts the in-flight LLM call through Vercel AI SDK v6 AbortSignal in ~200 ms. Both .catch handlers (selectSourceToIngest and ingestActiveFile) now call dismissProgress() so the persistent "Ingesting: " Notice clears on throw.

• 🛡️ Verbatim PDF transcriber prompt. The PDF→Markdown prompt is re-framed as OCR-style verbatim conversion with [illegible] / [figure: ...] / [equation: ...] anti-hallucination markers; small/local models that wrap output in ```markdown fences are auto-cleaned before cache write via unwrapFencedMarkdown(). Prompt centralized into `src/wiki/prompts/pdf.ts` alongside every other LLM-call prompt in the project.

• 🔍 Three-defense-layer cache filename safety. Physical filename = sha256(logicalKey).slice(0, 16) (Git short-hash style); logical key retains sha256:model:converterVersion semantics; converter hashes via new hashCacheKey() helper before cache.get/set. Fixes Windows ERROR_INVALID_NAME + POSIX unintended subpath when model contains / or :.

• 🌐 10 new i18n keys × 10 locales for the two new settings + PDF Ingest + Local PDF OCR Path (forcePdfSupportName, forcePdfSupportDesc, writePdfMarkdownToVaultName, writePdfMarkdownToVaultDesc, sourceRejectedPdfUnsupported, plus 5 unblocked PDF-related keys).

Changed

• Default behavior preserved. No breaking changes since v1.0.0. Old data.json without the new settings fields defaults to false, preserving cache-only behavior. The previously-planned sidecar-by-default approach was withdrawn in favor of cache-only before v1.25.0 ships.

• PDF dispatch lives in wiki-engine.ts. The separate pdf-ingest-orchestrator.ts was deleted; ingestPdfSource now feeds convertPdfToMarkdown result into analyzeSource via IngestOptions.contentOverride, reusing the entire existing Markdown ingest pipeline.

• forcePdfSupport is a universal escape hatch. Any non-native provider (custom, anthropic-compatible, ollama, lmstudio, deepseek, kimi, glm, gemini, openrouter) can attempt PDF conversion when the toggle is on. The endpoint decides; failures surface as a localized sourceRejectedPdfUnsupported Notice guiding the user to disable the toggle or switch provider. Switching the provider to a NATIVE one (anthropic / openai / bedrock-*) auto-resets the toggle to false. The FORCE_PDF_PROVIDER_IDS constant was deleted.

• 5 dead i18n keys removed across all 10 locales (old "PDF orchestrator" + sidecar-default language).

• LLMClient.createMessage gained abortSignal?: AbortSignal as an optional parameter. Existing client implementations ignore unknown params (graceful degradation).

Fixed

• ENOENT cache dir (Bug A). Obsidian adapter doesn't auto-create parent directories. PdfConversionCache.ensureCacheDir() walks path segments before mkdir.

• AI-SDK cause chain (Bug B). Vercel AI SDK v6 wraps provider rejections in error.cause.message. The pre-fix isPdfRelatedLlmError classifier inspected only error.message and missed the rejection phrase. inspectCauseChain() walks the cause chain up to 4 levels with cycle protection; classifiers consult both layers.

• Status bar didn't mirror Notice (Bug C). Clicking the status bar during PDF conversion didn't update text — fixed via double-callback pattern (Notice channel + text mirror in onProgress closure).

• PDF mid-flow cancel ineffective (Bug D). Two-layered bug: setup block re-initialized on re-entry overwrote AbortController, AND convertPdfToMarkdown didn't thread AbortSignal to the LLM call. Fixed with idempotency guard in wiki-engine.ingestSource + abortSignal threading through PdfConversionContext.

• pdf-cache never written (Bug E). Same root cause as Bug A but in the cache write path. ensureCacheDir() fix covers both directions.

• Stuck "Ingesting: " Notice (Bug H). When an interactive single-file ingest threw (network / vault IO / unexpected error), the persistent progress Notice stayed on screen until the next ingest. Both .catch blocks now call this.dismissProgress() after showing the error Notice.

• Markdown wrapper contamination in PDF output. Some local / small models (Qwen3.5-2B-MLX-4bit, Llama 3 8B Instruct, etc.) wrap their PDF-conversion response in markdown ... fences despite the system prompt forbidding them. unwrapFencedMarkdown() heuristic cleaner strips BOM → outermost ```markdown → outermost ``` → <output> → leading "Here is the converted Markdown:" preamble. Internal python ... blocks survive (regex is single-fence, outermost-only).

• PDF error classifier tightened. isPdfRelatedLlmError now requires BOTH a rejection verb (reject / not support / unsupported / invalid / not allowed) AND a PDF/media marker (pdf / application/pdf / file part / mediatype). Pre-fix classifier substring-matched on 'pdf' alone, causing transient 413 size-limit errors and Rust-serde "unknown variant file" schema rejects (no pdf keyword) to be misreported as "provider doesn't support PDF".

• TOC order corrected in 8 locale READMEs. CN, ZH-Hant, JA, KO, DE, FR, IT, ES previously listed 🛠️ Maintenance before 💬 Query & Feedback in the TOC, but the actual rendered order in those files was Knowledge Quality → Query & Feedback → Maintenance. All 10 locales now match. ES additionally had a stray - [](#) empty anchor + a wrongly-indented Arquitectura & Rendimiento at the top level (corrupted by a past bulk edit) — reconstructed from scratch.

Tests

• 2182 tests passing (165 files). +102 since v1.24.1.
• New tests cover:
  • 30+ PDF ingest end-to-end tests (provider gate, cache hit/miss, settings defaults, sidecar create/update, forcePdfSupport toggle, classifier, cause chain walking, status bar, cancel-mid-PDF)
  • 20 prompt invariant + unwrap helper tests (src/__tests__/wiki/prompts/pdf.test.ts)
  • 6 PDF error classifier regression tests (happy-path + 413/5xx/null-deref/generic-invalid guards)
  • 3 Bug D lifecycle tests (idempotency guard, AbortSignal propagation, dismiss on throw)

Contributors

• @green-dalii — PDF Ingest Level 1 architecture + cache + converter + metadata + prompt centralization + 9 PR3 follow-up bug fixes + lockfile fix for Obsidian build verification + 10-locale README sync + release workflow
• @AkaSakana — PR #286 design feedback adopted in v1.25.0 (cache key includes converterVersion; universal escape hatch). Kimi Files API + non-routine PDF provider dispatch deferred to PR4 → v1.26.0 MINOR target (we'll reply on PR #286 with timeline + invite AkaSakana to ship PR4 as a follow-up PR on top of v1.25.0)

Installation

From Obsidian Community Plugins:

1. Open Obsidian Settings → Community plugins
2. Click Browse and search for "Karpathy LLM Wiki"
3. Click Install, then Enable
4. (Or visit the Community Plugin website)

Manual installation:

• Download main.js + manifest.json + styles.css from this release
• Place them in <vault>/.obsidian/plugins/karpathywiki/

Configuration

• PDF Ingest: No configuration needed for native providers. For other endpoints, enable Force PDF Support in Settings → LLM Configuration → Advanced.
• Cache housekeeping: Runs automatically on plugin load and at the start of every batch ingest.
• Local PDF OCR path (Apple Silicon): Follow the Local PDF OCR Path guide to set up oMLX + Markitdown + Baidu Unlimited-OCR for fully-local PDF ingestion.
• Local models: See the Local Model Recommendations section for parameter-count tradeoffs and quantization guidance.

Full Changelog

Full changelog: 1.24.1...1.25.0

Issues closed

This release does not close any user-reported Issues. PDF Ingest (Level 1) is feature work tracked in Issue #218 + PR #286 (deferred to v1.26.0 MINOR).

Upcoming

• v1.25.1 PATCH (next cycle, ~2-3 days): DiskCache extraction, ledger optimization, provider-capabilities registry, HousekeepingTask registry, settings split. See ROADMAP.md.
• v1.26.0 MINOR: Kimi Files API + non-routine PDF providers (PR4, owned by AkaSakana via PR #286 follow-up) + wiki-engine.ts decomposition + Lint perf opening + #220 source-revision awareness.