PDF Source Ingest for v1.24.0 — open design discussion

[green-dalii]

PDF Source Ingest for v1.24.0 — open design discussion

Tracking #218 (filed by @A-NGJ). Confirmed real pain point, targeting v1.24.0 MINOR as a design-track discussion. PRs land on feat/pdf-support branch off v1.24.0-dev after we converge here.

Principle: every option is evaluated on total cost-of-ownership: maintenance burden + bundle cost + setup friction vs. user value. No path pre-selected — we're deciding together.

The problem

PDF sources (papers, reports, scans, exports) need manual pre-conversion to Markdown today. That breaks the "drop a source, let the wiki build itself" workflow. Goal: .pdf becomes first-class alongside .md as an ingest source.

What we already agreed on

1. readDocument() chokepoint in the LLM client — no per-provider branching in the ingest path
2. Extraction cache keyed on content hash — re-read only when the PDF actually changes
3. Graceful rejection for unsupported / scanned PDFs — clear Notice, never silent failure
4. Provider error surfaced (quota / unsupported / rate limit) instead of generic HTTP status
5. Provider support matrix declared upfront — no hidden assumptions

The first fork — where does the work happen?

Before picking libraries, decide topology. This is the largest ROI lever — wrong topology means rewriting the integration layer later.

Topology A — Plugin-native (this plugin does the extraction)

The plugin ships or calls its own extraction logic. Single install, single user action.

Pros	Cons
One user action; tight ingest integration	Bundle / dep maintenance burden on this plugin
We own the API surface	Another setting + i18n to maintain
Predictable UX across all providers	Updates here need Obsidian plugin review

Topology B — Delegate to an Obsidian PDF plugin

Require users to install + configure a community PDF plugin. Call its API from ours for conversion.

Pros	Cons
Zero new dependencies in this plugin	Cross-plugin coupling; version compatibility churn
Reuses mature community code	API surface may shift; not all plugins expose programmatic conversion
No bundle-size hit	Users must install another plugin — setup friction

Topology C — User-configured endpoint (generic service)

Plugin exposes one setting: "PDF extraction URL + API key". Users wire whichever service they prefer.

Pros	Cons
Zero bundled code / zero new deps	Setup burden on every user
Maximum flexibility across services	Error UX becomes service-specific and harder to localize
Future-proof against new services appearing	Discovering + comparing services is still the user's problem

Open question for the community: for an LLM-Wiki workflow specifically, which topology has the lowest TCO over a 12-month horizon? Anyone with cross-plugin-API experience please weigh in.

Extraction paths (assume Topology A or C)

Path 1 — Native LLM read

Anthropic / OpenAI (gpt-4o+) / Gemini natively accept PDFs in their message content blocks. The plugin already routes through @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/openai-compatible.

ROI	Notes
Highest for users on these 3 providers	Token cost on large PDFs; text-only providers (DeepSeek / GLM / Ollama / LMStudio) reject
Zero new code	Scanned / image-only PDFs need pre-OCR regardless

Path 2 — Hosted PDF-to-MD service (pay-per-page)

User supplies an API endpoint. Plugin POSTs the PDF, receives Markdown.

Service	Cost	Strengths	Weaknesses
Mistral OCR	~$2/1000 pages	SOTA quality; image extraction; multi-language	Recurring cost
Adobe PDF Services API	Pay-per-doc	Reliable native + scanned	Vendor lock-in; OAuth setup
Mathpix	Subscription	Formula / table heavy (STEM content)	Costly for non-STEM
MinerU (mineru.net)	Free tier + paid	Strong on Chinese / academic; 109 languages; structure preserved	Self-host option for free tier; paid for scale
Marker (datalab.to)	Per-doc	Tables / formulas / images; open-source backbone	Adds vendor between user and Marker

ROI	Notes
Best fidelity on scanned / structured PDFs	Recurring cost + API-key + service-availability risk each layer
Lazy setup — user opts in	Each service needs its own error-translation layer

Path 3 — Self-hosted / local service (zero per-page cost)

User runs an extraction service locally; plugin calls it. All have Docker / Python REST APIs suitable for Obsidian-side calls.

Service	Strengths	Weaknesses
MinerU (self-host)	Academic / multi-language; layout-aware	GPU-recommended; setup required
PaddleOCR (Baidu)	Lightweight; structured MD output; multi-language	GPU recommended (~8GB)
Baidu Unlimited-OCR (2026)	Long-document optimized; MoE 3B; low activation cost	Newer; less third-party validation
Marker (self-host, datalab-to/marker)	Benchmark leader on many docs	Python service; GPU strongly recommended
Microsoft MarkItDown	Lightweight; multi-format; very easy API	Less structure-aware than Marker / MinerU
PyMuPDF4LLM	Fast text-only extraction; tiny footprint	No OCR; text-PDF only
Docling (IBM)	Structured extraction	Python service; less battle-tested in plugin contexts
Local LLM + Ollama (Gemma etc.)	Maximal privacy	Slow; quality varies; needs pagination logic

ROI	Notes
Zero recurring cost	GPU / RAM / docker-compose is the user's setup burden
Full privacy — PDF never leaves the machine	Some require significant compute; quality variance per language
Plugin code stays small (just an HTTP POST)	User owns service availability — error UX needs to be diagnostic, not cryptic

Path 4 — Library bundled inside the plugin

Bundle a JS / WASM / Python library directly.

Library	Bundle cost	Offline	Quality
pdfjs-dist (~500KB)	Medium	Yes	Text only; no OCR
PyMuPDF4LLM via WASM	High	Yes	Text only
pdf-to-md style approaches	Varies	Yes	Quality varies by layout

ROI	Notes
Lowest setup friction — install only this plugin	Bundle bloat hits every user (including those who never ingest PDFs)
Always available offline	We own the maintenance of native deps

Path 5 — Delegation to an Obsidian PDF plugin (Topology B)

In Obsidian's community plugin ecosystem, several plugins already do PDF → Markdown conversion. The question for Topology B: can we call any of these from our plugin programmatically, rather than re-implementing extraction?

Plugin	Capability	Notes for our integration
Marker PDF to MD / OCR-AI / obsidian-marker	Calls Marker (self-host) or Mistral OCR for rich MD — tables / formulas / images	Most likely candidate for a programmatic API
pdf-to-md	Right-click "Convert to Markdown"; supports handwritten PDF via OCR	Needs verification on whether it exposes a public method
PDF Folder to Markdowns	Batch-converts PDF folders to embedded-MD files	Probably commands-only (no programmatic entry)
obsidian-extract-pdf (akaalias)	Basic reliable text→MD; preserves headings / lists	Lightweight; good fallback target
OCR Extractor / Text Extractor	OCR (Mistral / Tesseract) → MD; PDF + image support	Best for the scanned-PDF path

ROI	Notes
Zero new dependencies in this plugin	Cross-plugin version coupling
Reuses mature community-maintained extraction	API surface may shift; not every plugin exposes programmatic conversion
Zero bundle-size hit	Users must install + configure the chosen plugin — setup friction
Best fast-follow impact	Surface area to verify before commit: does any of these expose a stable JS API we can call from another plugin?

Excluded from this path (mentioned for completeness, not candidates): PDF++ (pdf-plus) is annotation-centric and does not do PDF→MD; it's a great UX companion but cannot serve as an extraction backend.

Cross-cutting decisions (any path)

Before committing to a path, agree on:

1. Cache key — contentHash (re-extract only on content change) preferred; mtime fallback acceptable
2. Image-only / scanned PDF UX — Notice + skip by default, or prompt user to configure a service?
3. Bundle budget — current main.js after AI-SDK v6 is ~3.2 MB; what's the PDF-related ceiling?
4. Long-term owner — @A-NGJ offered a prototype; does he want to drive v1.24.0 implementation?
5. Error UX responsibility — every failure path needs a Notice in the user's language; topology determines who owns this

Information we still need from the community

• Cross-plugin API surface — has anyone successfully called Marker / Mistral / pdf-to-md conversion from a different plugin (vs. invoked via command palette only)? If only commands are exposed, Topology B drops out.
• Real-world MinerU / PaddleOCR / Marker deployments — bundle size, startup latency, GPU/CPU trade-offs in practice
• Bundled-library TCO — anyone measured pdfjs-dist's contribution to plugin size in a real release?
• User-side preference — would you accept "install another plugin" for cleaner PDF UX, or do you want everything in one?

Next steps

• Comments on any of the above are welcome
• Once we converge on topology + first path, Add PDF support #218 will move from question to enhancement + help wanted
• PR can land on feat/pdf-support (branched off main after v1.23.0 ships on 2026-07-02)

---

cc @A-NGJ (submitter) — share your local prototype details (topology you picked, provider matrix, cache strategy) so we don't re-litigate decisions you've already made.

cc community — Obsidian users currently running MinerU / PaddleOCR / Marker locally: what's the real-world setup cost? Plugin developers who've cross-plugin-called PDF conversion APIs: what's the API stability story?

[DocTpoint]

Data point for the "cross-plugin API surface" question — I initially wondered whether this whole discussion was moot given Obsidian's own Web Clipper, so I checked before assuming: it isn't. Web Clipper explicitly does not extract PDF content. There's an open, unresolved issue on the official repo confirming this — obsidianmd/obsidian-clipper#646, filed Nov 2025, still open. Makes sense given its design: it's a browser-extension DOM→Markdown tool for capturing live web pages, not a PDF text/OCR extractor for files already sitting in a vault. Different problem shape entirely, so no overlap to worry about there — this discussion isn't reinventing something Web Clipper already covers.

That said, it does sharpen the case for Topology B specifically: since the most obvious "does Obsidian already solve this" candidate turns out not to, the value of verifying the dedicated PDF→MD community plugins already listed under Path 5 (do any expose a stable, callable API vs. command-palette-only) goes up — that's the one lever that could avoid bundling extraction code into this plugin at all. Worth prioritizing that verification before Topology A/bundling gets more design investment, since it's the cheapest way to falsify or confirm the lowest-TCO path.

[DocTpoint]

One personal-workflow data point that might matter for how broadly this needs solving: for my own use, I deliberately feed the wiki curated summaries rather than the original PDF — not as a workaround, but by design. I have to read the source paper first anyway, so writing a distilled note as part of that reading is close to free, and it keeps me in control of what actually enters the graph. Ingesting a 15-page Nature/Science article at full length would balloon the wiki with far more surface area than the extractable signal justifies — separate from token cost, a single source contributing that much raw text concentrates merge/hub exposure (see #216) more than a distilled note would.

Not arguing against solving PDF ingest for people who do want raw-paper-in — just flagging that "read it, then write a short note" may remain the better default for some workflows even once native PDF support exists, rather than something PDF support is expected to replace.

[A-NGJ]

Thanks for the writeup @green-dalii . As requested, here are my details of my local prototype.

TL;DR: Topology A + Path 1 (native LLM read), zero new dependencies, built as three pure-addition phases on the pre-AI-SDK codebase.

Topology picked: A, Path 1

I went with plugin-native extraction via the provider's own document support - no bundled lib, no external service, no cross-plugin coupling. The rationale in TCO terms:

• Zero new deps / zero bundle hit. PDF bytes are read with vault.readBinary(), base64-encoded with chunked btoa() (32KB chunks to avoid String.fromCharCode RangeErorr on large files). No pdfjs, no WASM.
• Zero serup friction. for users already on Anthropic / OpenAI / Gemini-compatible endpoints.
• It doesn't close Paths 2/3. Everything goes through one checkpoint (below), so a hosted or self-hosted extraction endpoint can later slot in behind the same seam as a fallback for text-only providers, without touching the ingest path again.

Architecture

1. LLMClient.readDocument(pdfBase64, opts) - optional method on the client interface, implemented per provider:

• Antropic / AnthropicCompatible -> Messages API document content block: {type: 'document', source: {type: 'base64', media_type: 'application/pdf', data}} + text prompt.
• OpenAICompatible -> Chat completions file block: {type: 'file', file: {filename, file_data: 'data:application/pdf;base64,...'}}.
• Both go through a shared sendDocumentRequest() wrapper: withRetry (3 attempts, backoff) for 429/5xx, plus provider-error enrichment. createMessage / createMessageStream are untouched.

2. WikiEngine.readSourceContent(file) - the single text-vs-PDF branch point. Text -> vault.read() (byte-identical to before); PDF -> cached extractPdfViaLLM(). All three read sites in an ingest (pre-ingest gate, source analyzer, summary page) go through it, so a PDF is sent to the model at most once per ingest.

3. Activation is a one-line allowlist change - adding pdf to COMPATIBLE_SOURCE_EXTENSIONS. Until that line loads, the entire extraction path is inert, and FileSuggestModal picks up .pdf automatically once it loads.

Provider matrix

Provider	PDF	How
Anthropic / Anthropic-compatible	✅	Messages document block
OpenAI-compatible (gpt-4o+ class)	✅	Chat Completions file block
Ollama / LM Studio	❌ fast-fail	Advisory pre-check, no LLM call wasted
DeepSeek	❌ fast-fail	No document block in Chat Completions (V4 added images, not files)
Kimi (Moonshot)	❌ fast-fail	PDF only via their separate Files API (purpose="file-extract"), not chat content blocks
MiniMax	❌ fast-fail	image_url/video_url blocks only, no document input
GLM (Zhipu)	⚡ optimistic	Model-dependent: GLM-4.5V+ vision models accept files in chat completions; runtime backstop decides per model
Gemini (Flash/Pro)	⚡ optimistic	Models fully support PDF (native API, up to ~1000 pages), but the plugin talks to the AI Studio OpenAI-compat endpoint, which documents only image/audio inline blocks - type: 'file' is unverified there. Confirmed working via LiteLLM proxy, which translates the file block to Gemini's native format
OpenRouter / custom / unknown	⚡ optimistic	Runtime backstop — new multimodal endpoints work day one

Two layers: providerSupportsPdf() is a cheap advisory pre-check (text-only providers rejected before any network call, surfaces as an unsupported-pdf rejection reason with a localized Notice). The backstop is runtime: a 400-class response from readDocument is classified by isDocumentUnsupportedError() into a PdfUnsupportedError sentinel -> same graceful rejection path. Unknown providers stay optimistic so new multimodal endpoints work day one.

Cache strategy - path@mtime, per-ingest (deliberately not contentHash)

What's implemented: a per-ingest cache keyed ${path}@${mtime}, cleared when the ingest ends. Its job is deduping the three read sites within one ingest (gate, analyzer, summary page) so a PDF is sent to the model at most once per ingest — and since it never outlives the ingest, mtime is sufficient and free off TFile.stat, while contentHash would require reading the full binary anyway.

Error UX

One fix worth flagging: Obsidian's requestUrl throws on 4xx without the response body, so users would see a bare "status 400". On readDocument failure the prototype re-fetches once via window.fetch (5s timeout, only on the failure path) to recover the provider's actual message — so the Notice reads e.g. status 400: This model does not support document input instead of an opaque code. Same pattern as the v1.22.5 Test Connection fix. Three new i18n keys (pdfReadingInProgress, pdfExtractedTextHeading, sourceRejectedPdfUnsupported) across all 10 locales.

Scanned/image-only PDFs: quality depends on the model's vision capability; if extraction comes back empty, the existing pre-ingest empty-content gate rejects it with a clear Notice — never a silent failure. Corrupt/password-protected PDFs propagate as ordinary errors (not misclassified as "unsupported").

Output

The source summary page for a PDF embeds the model's transcription under a localized ## Extracted Text heading - since unlike a text source, the original isn't otherwise readable in the vault graph.

One caveat for v1.24.0

My prototype is built on the current custom-client codebase (llm-client.ts), not the AI-SDK migration mentioned in the discussion. The design ports cleanly — readDocument maps to AI SDK file parts in the message content, and the chokepoint/cache/rejection layers are provider-agnostic — but the body-builder layer would be rewritten against @ai-sdk/* rather than merged as-is.

Contributing

Happy to share the branch / open a draft PR against feat/pdf-support once v1.23.0 ships so this can serve as the reference implementation for the discussion. I could take ownership depending on the 1.24.0 release timeline. Let's say that 1.23.0 lands today, I would need a reasonable 3+ week window from there to finish the implementation.

[A-NGJ]

@DocTpoint I use LLM-wiki mainly to ingest technical documentation and build a knowledge base about a project/organization. In that use case, reading every source and summarizing it beforehand would be tedious. What ingesting the entire PDF gives me is an index that enables fast lookup and concept-linkage. I don't need to understand the whole document the first time I encounter it — I need its content stored and processed in a way that lets me easily search through it when I need a specific piece of domain knowledge (or point an AI agent at it).

On prioritizing the Topology B verification: agreed it's cheap and worth doing, and I'd welcome the result either way. But I'd flag that even a positive result (some plugin exposes a callable conversion method) doesn't resolve B's structural problems: Obsidian has no inter-plugin API contract — you reach into app.plugins.plugins['id'] and call whatever's there, untyped, unversioned, with no way to declare the dependency or get notified when the user disables it. And the plugins that do high-quality PDF→MD (obsidian-marker, the Mistral OCR ones) are themselves wrappers around an external service, so B doesn't remove the API-key/service setup cost — it adds a coupling layer on top of it. My fuller argument is in the main reply, but the short version: the legitimate need behind B (a non-LLM extraction route for text-only providers) is served better by Topology C, which gives the same zero-bundle benefit against a stable HTTP interface instead of another plugin's internals.