Architecture: Personalized PageRank (PPR) graph engine — v1.23.0 direction (consolidates #117, #157, #175)

[green-dalii]

The shape of the problem

We have a wiki that is, structurally, a graph. Every [[wiki-link]] between pages is an explicit assertion by the LLM at ingest time that these pages are related. We are sitting on a graph that already encodes exactly the semantic relationships we'd otherwise need an embedding model to compute — and currently we are not using it for any of the three things it should power.

This is a tracking issue for the v1.23.0 direction. It supersedes the embedding-layer proposal in #175 and consolidates the hub-detection (#117), link-distinctiveness (#157), and Query-Wiki-relevance work into a single graph-native engine. It is the discussion thread for the architecture; PRs will follow once we agree on the primitive.

The proposal in one sentence

A single Personalized PageRank engine (Haveliwala, "Topic-Sensitive PageRank", WWW 2002) over the existing [[wiki-link]] graph, computing all three of the signals we need from one iteration.

If that sounds abstract — picture your wiki as a small town. Every wiki page is a house. Every [[wiki-link]] is a well-worn footpath between two houses. The houses that the town's residents visit most often are the important ones. Now imagine you walk into town and ask "where's the bakery?" — you start at the bakery, and every minute you have an 85% chance of following a random footpath to a neighbor, and a 15% chance of teleporting back to the bakery. After walking for an hour, the houses you've visited the most are the ones most relevant to the bakery. That's PageRank. "Personalized" just means we start the walk from the bakery, not from a random house — so the answer is "what's near the bakery?", not "what's important in town?"

That's exactly what we need for a wiki question: the user asks "what is Kardiologie?" — we seed the walk at Kardiologie, run for 50 iterations, and the pages that accumulate the most visits are the ones we should put into the LLM's context. No embedding model, no provider lock-in, no persistent cache, no API cost.

Why not embeddings (reprising the decision in #175)

Embedding-based relevance was the original proposal in #175. After discussion we rejected it on three grounds:

1. Provider lock-in. Only OpenAI natively supports /v1/embeddings. DeepSeek, GLM, MiniMax, Ollama, OpenRouter — the providers most of our users actually run — don't. A shared infrastructure built on a single-provider endpoint locks out the majority of the user base.
2. Persistent state burden. A per-page embedding index for a 2000-page vault is roughly 12 MB of JSON that has to be versioned, migrated, invalidated on model change, and held in memory.
3. Mathematical equivalence for our use case. Haveliwala's 2002 paper proves that Personalized PageRank over a link graph produces a ranking mathematically equivalent to cosine distance over the graph's adjacency-matrix representation — i.e., it gives you what an embedding would have given you, but the graph is already the data the embedding would have learned from. We don't need to project; we can walk.

The LLM stays the verifier. Cosine is noisy in dense mechanistic clusters, and there's no natural threshold for "is this related enough" — we use PPR as a ranker / prefilter, not a judge, exactly as #175 proposed. The LLM's job is unchanged; we just give it better candidates.

The single primitive, four consumers

One PPR engine (src/core/ppr.ts, ~80 LOC, O(V+E·iter), pure JS, no deps) provides:

#	Consumer	Existing Issue	How PPR answers it
1	Hub detection	#117	Top-N pages by inDegree + PageRank. Add @DocTpoint's local-clustering-coefficient retirement signal from the 2026-06-23 reply — hubs retire when their neighbors have interlinked into chains rather than radiating out from a center.
2	Link distinctiveness	#157	distinctiveness(P→T) ≈ shared(P,T) / PageRank(T). Replaces @DocTpoint's embedding-cosine formula with a link-overlap ratio that has the same ranking behavior on the existing graph.
3	Query retrieval (replaces the old ROADMAP §Query Engine Evolution Tier A)	(internal)	Seed PPR at the page the user mentioned, take top-k by score, hand to the LLM. No more lex-match on index.md.
4	Dead-link hub check	#197	A dead link is more likely an honest forward reference when the target would be a hub (high in-degree, high PageRank). A dead link is more likely a typo / retarget candidate when the target is a leaf.

Tier B redesign (zero LLM)

The old ROADMAP Tier B called for a per-query LLM call to generate a "summary" for each page. We are cancelling that.

Every wiki page already has a ## Description section (entity pages) or a ## Definition section (concept pages). It was generated by the LLM at ingest time. We will extract it on demand, at query time, with a ~30-line pure function that:

1. Parses the markdown to find the section header
2. Slices the section content (up to the next ## or end of document)
3. Strips [[wikilink]] syntax to its display text (so the LLM gets clean prose, not raw links)
4. Truncates at a sentence boundary around 300 characters

No new frontmatter field, no LLM call, no schema migration, no compatibility issue with old wikis. The summary the LLM sees in the query context is the description the LLM itself wrote at ingest — the most reliable source of "what is this page" that exists.

What this does NOT include (deferred)

• Architecture: a shared embedding "structural awareness" layer behind #117, #157, dedup, and #164-grounding #175 embedding layer: superseded by PPR (the [[link]] graph already encodes the relationships).
• ROADMAP §Query Engine Evolution Tier C (separate in-memory knowledge graph + traversal): superseded by PPR multi-seed iteration. No need to maintain a parallel graph structure.
• ROADMAP §Query Engine Evolution Tier D (agentic loop with tool calls): deferred to v1.25.0+. Requires LLM function-calling support across the full provider matrix; currently only Anthropic / OpenAI / Gemini are stable, Ollama is partial, and the rest is evolving. Out of scope until the matrix stabilizes.
• Propagate source-note frontmatter aliases: to the wiki pages they describe #185, Enhancement: Obsidian Bases for wiki index management #184, Feature: in-place batch ingest queue for the Ingest button — avoid moving source notes into a staging folder #130, Enhancement: adopt Obsidian Keychain (SecretStorage API) #182, Nested tags for entity/concept pages based on source note tag hierarchy #91, Feature: Add explicit event marker/type and enhanced local indices for improved LLM-Wiki structuring #112, 'auto' granularity setting - maps granularity to file size based on an equation #168, Is it possible to create multiple wikis on various topics? #142: independent features; not in this thread.

What we are asking

This is the discussion thread before we cut code. Specifically we want feedback on:

1. The primitive choice. PPR vs embedding — is the equivalence argument convincing, or are we missing a case where embeddings add signal that link structure cannot capture (e.g. cross-lingual, semantic similarity on a single new page that hasn't been linked yet)?
2. The consumer list. Are these four the right consumers, or is there a fifth we should fold in?
3. Cold start. For a new vault with few pages, PPR on a sparse graph is mostly zeros. Lex match as a fallback? Force-ingest with a "seed hub" pass first? Or just accept that very small vaults get the current lex behavior?
4. Performance budget. 2000 pages × 50 iterations × avg outDegree 5 ≈ 500k floating-point ops per Query. Should we run PPR on a Web Worker to keep the Obsidian main thread responsive, or is in-thread < 100ms acceptable?
5. Caching strategy. PPR scores depend on the query seed and the graph state. The graph changes on every ingest. Options: (a) recompute on every Query (simple, ~100ms), (b) cache per-seed and invalidate on ingest affecting that seed's neighborhood, (c) cache the per-node PageRank and only run the seed-specific restart vector at query time.

Anyone who has thoughts, especially from #117, #157, #175, #164, and the Lint / Query code paths, please weigh in. We will not start implementation until this thread has had at least a few days of discussion.

cc / ping

• @DocTpoint — author of Feature: Graph-based hub detection for automatic domain tag inference #117 (hub detection) and Proposal: replace hub-tag link stripping with embedding-distinctiveness (fixes over-stripping of legitimate broad→hub links) #157 (link distinctiveness); your 2026-06-23 reply on Feature: Graph-based hub detection for automatic domain tag inference #117 introduced the clustering-coefficient retirement signal. PPR with degree + clustering should give you exactly the direction you described, with the temporal axis covered by a separate coefficient instead of entangled with detection.
• @Indexed-Apogrypha — author of CRITICAL BUG: plugin creates hallucinated concepts and entities when ingesting empty notes #164 (empty-source hallucination); Fix Dead Links fabricates AI-expanded stub pages for unresolvable links — reintroduces the blank-source hallucination #174 prevents #197 is your finding that fixDeadLink reintroduces the same hallucination class. The dead-link hub check (consumer Generated wiki is in Chinese, even when choosing English #4) is the seam where the two meet.
• @alfred1137 — author of Enhancement: Obsidian Bases for wiki index management #184 (Obsidian Bases index). PPR may obviate the need for a separate index file at all; the graph is the index. Worth comparing.
• @Linate — author of feat: One-click apply for schema suggestions with auto-backup #97 (the schema one-click apply that just shipped in v1.22.0). You have an eye for "what does the user actually feel" — does "Query Wiki gets the right answer without me writing an index file" resonate?

References

• Haveliwala, T. H. (2002). Topic-Sensitive PageRank. WWW 2002. (PDF) — the original PPR paper, including the equivalence-with-cosine argument.
• Page, L., Brin, S., Motwani, R., Winograd, T. (1999). The PageRank Citation Ranking. — base PageRank.
• Jeh, G., Widom, J. (2003). Scaling Personalized Web Search. — multi-seed PPR for batch queries.
• Backstrom, L., Leskovec, J. (2011). Supervised Random Walks. — the original "personalized random walk for predicting edges" paper; relevant if we later want to learn edge weights from user feedback.

[GioiaZheng]

I like PPR as the primitive for a wiki where links are intentional, but I would not lean too hard on the embedding-equivalence argument. PPR over the link graph captures relation-by-link behavior; embeddings can still add signal for unlinked, newly added, cross-lingual, or vocabulary-shift pages. So I’d frame PPR as the default graph-native ranker, not as a full semantic replacement.

For cold start, I’d keep a hybrid fallback: lexical/title match seeds first, PPR expands from those seeds only when the graph has enough edges. A simple guard like min_edges/min_outdegree before using PPR could avoid the “mostly zeros” case.

For evaluation, a small fixture of query -> expected pages would be useful before implementation. Compare lex-only, PPR-only, and lex-seeded-PPR on recall@k and noise in the final context. That would make the primitive choice easier to defend than the theoretical equivalence alone.

[DocTpoint]

Picking up on cold start (#3), the #1 equivalence question, and performance/caching (#4/#5).

The hub lifecycle has two ends; PPR detection only covers the middle. Scaffolding goes up before construction and comes down after:

• Birth (cold start). On an empty vault there's no structure to detect — PPR is all zeros, and no algorithm can know the user's domains from zero data. So don't detect; seed. Cleanest form is a guided onboarding step: on first run, ask the user to pick ~5 source notes from different domains and ingest those first.
  • "Different domains" is load-bearing — the value isn't 5 notes, it's 5 spread notes. They give the cold graph distinct basins instead of one blob, which is what PPR needs to make domains separable.
  • Honest scope: those seeds create regions with real sourced content, not finished hubs — a hub crystallizes only once later notes point at a shared center. Two steps: (1) ingest 5 diverse notes → enough spread for PPR to leave the zero state; (2) optionally let the user mark each domain's central concept as an anchor. That anchor is the scaffolding.
  • This is complementary to @GioiaZheng's lex-seeded hybrid, but aimed a layer deeper: lexical seeding still needs something already in the vault to match against — on a genuinely empty vault there's nothing, and the missing input isn't text-similarity, it's which domains exist. That's information only the user has, which is why the seed step is a question to the user, not an algorithm.
• Middle. Now there's structure; degree + PageRank detection identifies hubs as domains grow.
• Death (retirement). The clustering-coefficient signal fires when a domain's children interlink into chains → retire the anchor. This is what makes seeding safe: without it, planted anchors ossify forever; with it, they're honest temporary scaffolding.

On #3 — I'd gently push back on a purely algorithmic cold start. The missing input on an empty vault is which domains exist, and only the user has it. A guided "pick 5 seeds" step is cheaper, more correct, and gives a real UX answer (re: @Linate): Query Wiki works on day one because you planted 5 seeds.

On #1 — +1 to @GioiaZheng: the unlinked / newly-added / cross-lingual page is exactly where PPR is blind (a fresh page scores ~0 however central it is semantically). Worth adding that the seed step doubles as the pragmatic mitigation for the new-vault slice of that gap — you don't wait for the topology to know a page is central, you plant it.

On #4/#5 — one option worth tabling: Monte Carlo PPR. Your sketch is power-iteration: compute the walk's stationary distribution deterministically, ~50 sweeps over the whole graph (the 500k-ops figure). The alternative is to sample the walk rather than solve it (Fogaras et al. 2005, Towards scaling fully personalized PageRank): simulate K short random walks from the seed, count visits, take top-k.

• Cost scales with walk length × K, not with V — a 2000-page vault costs the same per query as a 200-page one; Generated wiki is in Chinese, even when choosing English #4's budget concern largely dissolves for top-k.
• Walks are local to the seed's neighborhood, so there's little to cache — 🎉 Release v1.6.5 — Wiki Output Language Fix + Cumulative Quality Update (v1.6.2–v1.6.5) #5 mostly goes away, recompute is the cheap path.
• Embarrassingly parallel — fits the Web Worker idea.

The cost is that it's approximate (sampling noise), but that's a non-issue under your own framing: PPR is a ranker/prefilter, not a judge — top-k is robust to the noise, exact scores were never needed. The retirement signal stays orthogonal either way: clustering is a seed-independent global node property, computed once and invalidated on local ingests, however you run the query-time walk.

On the build itself — I won't pre-empt the discussion, but the clustering-retirement signal is a small, self-contained piece: a pure graph property, seed-independent, so it doesn't block on the PPR core, and I already have a reference implementation. Happy to own that as the separate commit you mentioned once the direction settles — and to keep reviewing the framing in the meantime.

[GioiaZheng]

Thanks @DocTpoint, this distinction makes sense to me. I agree that the fully empty-vault case is different from the sparse-but-not-empty case.

I’d frame them as two separate cold-start paths:

• empty vault: guided user seeding, because the missing signal is domain intent, not ranking
• sparse existing vault: lexical/title seeding first, then PPR expansion once there are enough edges to make the walk meaningful

In that sense, guided seeds feel less like a retrieval fallback and more like graph initialization / temporary scaffolding. Lexical seeding is still useful, but only after there is already something in the vault to match against.

That also gives a clean evaluation matrix across wiki lifecycle stages:

1. empty vault with guided seeds
2. sparse vault with lex-seeded PPR
3. mature vault with graph-first PPR

Comparing those separately would avoid judging one strategy against the wrong stage of the wiki lifecycle.

Monte Carlo PPR also seems like a good fit if the goal is top-k context selection rather than exact global scores. The approximation noise feels acceptable as long as the LLM remains the verifier and the eval checks recall@k / context noise instead of score stability.

[green-dalii]

Summary of where we agree (and what I'm hearing)

Thank you both — this is exactly the kind of discussion I was hoping for. Concise summary of the convergence so far so we're working from the same map.

Consensus

• PPR is the right default graph-native ranker, not a full semantic replacement. @GioiaZheng's correction stands: embedding-equivalence is the upper bound of PPR's behavior, and the cases where embeddings genuinely add signal (unlinked newly-added pages, cross-lingual, vocabulary-shift) are real, not theoretical. We frame PPR as the ranker + a hybrid fallback for the cases it genuinely can't see.
• Cold start is two distinct problems. Empty vault = UX problem (which domains exist), not algorithm problem. Sparse vault = algorithm problem with a clean threshold-based fallback. @DocTpoint's hub-lifecycle framing (birth / middle / death) is the right structure to organize the work — and the "death" end (retirement) is independent of the rest, so we can ship without it first.
• Monte Carlo PPR wins over power-iteration for our use case (Fogaras 2005, Towards scaling fully personalized PageRank). Cost is O(K×L) — independent of |V| — so a 20000-page academic vault and a 200-page personal vault cost the same per query. Embarrassingly parallel → Web Worker, no main-thread block on Obsidian UI. Top-k ranking is robust to the sampling noise; exact scores were never needed because PPR is a ranker, not a judge.

Maintainer decisions

Three decisions I'm taking into v1.23.0 scope. Calling them out so we can keep this thread focused on the open questions.

1. Adopt Monte Carlo PPR as the v1.23.0 primitive. Pure-function src/core/monte-carlo-ppr.ts (~30 LOC), wrapped in a Web Worker, fed by the existing [[wiki-link]] graph. No embeddings, no persistent cache, no provider lock-in.

2. Adopt @DocTpoint's correctRelatedLinkPrefixes as a precedent for hub retirement: same core/ location, same pure-function shape, same [[section-driven]] scoping. Defer hub retirement to v1.24.0 — the only consumer that needs it is tag inference (Feature: Graph-based hub detection for automatic domain tag inference #117), and tag inference itself is deferred to v1.24.0+ per the discussion in Feature: Graph-based hub detection for automatic domain tag inference #117. (Confirms @DocTpoint's offer to own the separate commit when we get there — no rush.)

3. Hybrid guard (lex-match fallback) is in scope. ~10 lines: a threshold on min_pages / min_edges below which we lex-match. Cheaper than the alternatives and cleanly tested. Default thresholds open for discussion below.

ROI for the proposed v1.23.0 scope

Roughly:

Module	LOC	Effect	Status
core/monte-carlo-ppr.ts	~30	Query Wiki + hub detection + related-page all collapse onto one primitive	high ROI — five separate features (#117 detection, #157 distinctiveness, ROADMAP Tier A query retrieval, ROADMAP Tier C traversal) close to one engine
core/hub-detection.ts	~50	Top-N by inDegree + MC-PPR score	medium-high
core/section-extractor.ts	~30	Zero-LLM Tier B (parse ## Description / ## Definition at query time)	high
core/link-distinctiveness.ts	~40	shared(P,T) / MC-PPR(T)	medium — only useful if we want per-edge signals; can defer
Web Worker wrapper	~30	Off-thread PPR for non-blocking	medium
Eval fixture (3 test vaults, recall@k vs lex-only)	~half day	Defends the primitive choice; required before merge	high
Hybrid guard (lex fallback)	~10	Cold-start safety net	medium
Hub retirement (clustering)	~50	Cleanup only needed for tag inference (v1.24.0)	defer
Total v1.23.0 estimate	~5 days	One consumer collapse + 1 P0 fixup

Skip-link-distinctiveness for v1.23.0 if scope is tight — it's a refinement, not a closure.

Open questions I'd still like to nail down

1. @GioiaZheng, on the "embeddings as opt-in enrich" idea — do you have a sense of which providers our users would actually run that support a thin embeddings endpoint? If it's basically "only OpenAI" then it's the same provider lock-in argument and we should defer embeddings to a v1.25+ "if user provides an embedding endpoint" feature. If Anthropic / Gemini have added a stable lightweight embeddings endpoint I'm not aware of, the calculus changes.

2. @DocTpoint, on the 5-seed guided onboarding — for the first-run UX, would you prefer this as a Modal (forced blocking — user has to seed before any ingest works) or as a Notice + non-blocking "seed now / later" prompt? I'd lean non-blocking (more aligned with Obsidian's deferral style) but you have the 900-page-vault empirical data, so your call.

3. @GioiaZheng, on the cold-start threshold — your "min_edges / min_outdegree guard" idea is the right shape. Two questions on numbers:

   • What's the empirical threshold below which PPR top-5 recall drops below, say, 80%? I'm guessing min_pages=50 is the right order of magnitude from the link-density data you mentioned, but real numbers would let me default it correctly.
   • Should the fallback be pure lex-match (current Query Wiki behavior) or lex-seeded PPR (PPR from a lex-matched seed page)?

4. Eval fixture scope — would a single shared fixture vault in this repo (say src/__tests__/fixtures/wikis/sample-50page/) work for everyone, or do you have private vaults you'd want to test against that don't fit in a public repo? If shared, the fixture needs to be CC0 / public-domain to avoid licensing issues with the LLM-generated content.

Friendly note

I don't have a hard timeline on this — the v1.22.0 release closed yesterday and the next sprint (v1.22.1 PATCH) is collecting in-the-wild P0 fixes (#187 fix in PR #200 just landed from @DocTpoint, #197 dead-link fabrication going into the same PATCH, #196 Query Wiki side panel from @YounianC under review). v1.23.0 is the sprint after. So the v1.23.0 implementation can start whenever the eval fixture questions above are answered, and the implementation itself is ~5 days.

If anyone wants to drive any of these modules, the door is open — same pattern @DocTpoint set with #200 (one self-contained PR per module, with a reference impl if you have one). The PR #200 pattern (PR description naming the two named test cases that catch the regression) is exactly what I want for the rest of v1.23.0.

Thanks again — this is the cleanest version of the hub-detection / PPR / query-retrieval discussion I could have hoped for.

[DocTpoint]

On Q2 (Modal vs Notice) — non-blocking, but I'd reframe the axis.

Agreed on non-blocking: a forced modal on install fights Obsidian's deferral style, and you lose nothing by letting the user arrive at the step on their own terms.

But Modal-vs-Notice is really an information-capacity axis, and for the actual goal — a first run that's self-explanatory, no docs required — both ends are weak:

• Modal is self-documenting but heavy; it interrupts before the user has any context for the question it's asking.
• Notice is unobtrusive but too thin to teach. A toast can nudge ("seed your wiki"), it can't guide — it vanishes and the user is back to square one.
• Settings fields fail the no-docs test by construction: a first-timer doesn't know to open settings. That's where power users go, not where newcomers get onboarded.

The form that actually satisfies "understand it without a manual" is a first-run welcome note — a real markdown note the plugin writes to the vault and opens in a tab on first launch. It's non-blocking (just a tab you can close), it's self-explanatory (the note is the instructions, by example), and for a wiki plugin it can pull double duty: pre-fill it as the founding note —

List the domains this wiki should cover, one per line: cardiology, immunology, psychology, …

The user edits one note — the most native action in Obsidian — and that content becomes the domain declaration. Those domains then seed the controlled tag vocabulary, which is what gives later-ingested notes distinct basins for PPR to separate. The note seeds the vocabulary; real ingested content seeds the graph.

Two things worth flagging:

• This is the empty-vault answer, complementary to the 5-seed step — not a replacement. The 5-diverse-notes onboarding assumes notes already exist (the sparse case); the welcome note handles the genuinely empty vault, where the only missing input is which domains exist, and a note the user types is the friendliest way to capture exactly that.
• It must seed vocabulary/tags, never fabricate pages. A user-authored template note is fine — it's their own words. The line not to cross is auto-generating empty domain pages from those fields; that's the stub-fabrication pattern Fix Dead Links fabricates AI-expanded stub pages for unresolvable links — reintroduces the blank-source hallucination #174 prevents #197 just closed. The founding note feeds the tag prior, it doesn't materialize content.

So my vote on Q2: non-blocking, delivered as an editable welcome/founding note rather than a toast or a settings panel — one intuitive action (fill in a note) that does onboarding, domain declaration, and vocabulary seeding at once.

[DocTpoint]

On Q4 (eval fixture) — shared synthetic fixture is the right call; I'm a no on contributing a real vault.

For the public repo, a small CC0 synthetic fixture (your sample-50page sketch) is exactly right: reproducible, lives in CI, and anyone can rerun the recall@k-vs-lex-only numbers. That's the role a shared fixture should play — the regression oracle, owned by the repo.

My vault is not a candidate for that, and I'd rather be clear-eyed about why than leave it ambiguous: it's a working clinical knowledge base — German-language medical content authored from a physician's notes. Public-domain release isn't on the table in any form, anonymization included; medical material in lay hands carries real-world risk that a fixture's value doesn't justify. So please don't size the plan around it being shareable.

What is shareable, if it ever helps, is pure topology — page/edge counts, degree distribution, clustering coefficients, link density at ~900 pages — numbers that carry zero content and reconstruct nothing. If a real mature-vault density check would help sanity-check the min_pages threshold (Q3) against something bigger than a 50-page synthetic, I can run it privately and report only the aggregates. One scheduling note: I'm only intermittently at a real machine for the next ~5 days, so treat this as async — ping me and I'll run them when I'm back. Nothing here is on your critical path anyway: the recall curves can't be produced until the v1.23.0 PPR core exists, so the synthetic fixture is what the merge gate actually needs now.

[green-dalii]

You're right to call that out, @DocTpoint — apology owed.

I should have caught it before sending. Privacy is one of this project's core commitments (data stays in your vault, period), and extending that posture to how we build the test suite — not just how the runtime handles data — should have been my first instinct. The "shared fixture" framing slipped through because I didn't actually re-read the AI draft before posting. That's on me. Sorry.

Going forward, the public repo gets a synthetic or CC-licensed fixture only — no real vault content, ever. I'll spec the build script as part of the v1.23.0 work. If you're ever up for sharing a one-shot aggregate topology report (page count, edge count, degree buckets — zero content), that would sharpen the min_pages threshold against a 900-page mature shape. But no pressure — the synthetic fixture gets us a merge gate; real-vault feedback can wait for v1.23.x.

Thanks for the steady work on this thread. The hub-lifecycle framing and the Monte Carlo argument are the spine of the v1.23.0 plan, and they came from digging past the first easy answer. That's the kind of contribution that makes a project better — and it's worth saying out loud.

[GioiaZheng]

On Q1/Q3:

Q1: embeddings as opt-in enrich. I would keep them out of the v1.23.0 core. The provider picture is better than "only OpenAI" now, but still not a clean common denominator: OpenAI, Gemini, Ollama, and OpenRouter expose embedding endpoints; Anthropic's Claude API docs still list Messages / Batches / Token Counting / Models / Files / etc., not embeddings. More importantly, even where embeddings exist, the contract differs: task formatting, dimensionality controls, batch shape, local model availability, cost, and persistence semantics.

So I would make the future shape:

• PPR / lex remains the default path.
• Embeddings are an opt-in enrichment layer behind a capability probe: endpoint, model, dimensions, and maybe task mode.
• Cache key includes vault graph/content hash + model + dimensions + provider.
• Use it only for the cases PPR cannot see: unlinked/new pages, cross-lingual, vocabulary shift, maybe alias discovery.

That keeps provider support from becoming a v1.23 dependency.

Q3: threshold/fallback. I would not use min_pages=50 as the main guard by itself. Page count is a weak proxy; edge density and seed locality matter more. A conservative initial guard could be:

• if pages < 25-30, pure lex/title match;
• if edges < pages or the lex seed has degree == 0, pure lex/title match;
• if a credible lex/title seed exists and has neighbors, run lex-seeded MC-PPR;
• once the graph is dense enough, graph-first MC-PPR with lexical/title matches as seed/boost.

So my answer on "pure lex vs lex-seeded PPR": lex-seeded PPR when there is a real seed with a neighborhood; pure lex only when the graph is too sparse or the seed is isolated. The fallback should be a small cascade, not one global mode.

For defaults before eval, I would start low and conservative:

• min_pages = 30
• min_edges = 30 or edges/pages >= 1.0
• seed_degree >= 1
• optionally require the largest weak component to contain >50% of pages before calling it graph-first

Then tune those with the synthetic fixture. If @DocTpoint can later report only topology aggregates from the 900-page vault, that is enough to sanity-check whether the defaults are absurd without needing private content.

Provider docs I checked quickly: OpenAI embeddings, Gemini embeddings, Ollama /api/embed, OpenRouter embeddings, Claude API overview.

[DocTpoint]

@green-dalii — no apology needed, honestly. It was an AI-draft slip on a detail, not a posture; the moment it surfaced you extended the privacy commitment from runtime to test suite without hesitation, which is the part that matters. Appreciated all the same.

Here's the one-shot topology aggregate — pure structure, zero content, no page titles. Live wiki graph of the vault.

Method (so the numbers are interpretable): nodes = wiki page files (entities + concepts + sources); edges = deduped [[wiki-link]]s resolving to another page, directed. Bare [[X]] resolves to the entity/concept page, [[sources/X]] to the source page. The source/entity duality (an ingested note has both a sources/ page and an entities/concepts page) is kept as two distinct nodes, mirroring how the link graph actually resolves.

Nodes:                        910   (entities 349 / concepts 479 / sources 82)
Directed edges (deduped):     5862
edges / node:                 6.44
mean total degree:            12.88
isolated (degree 0):          1    (0.1%)
out-degree 0:                 12   (1.3%)
in-degree 0:                  46   (5.1%)
largest weak component:       909  (99.9%)   [2 components total]
avg local clustering (deg>=2): 0.537
unresolved links (dead/forward refs): 154

total-degree:  0:1   1-2:69   3-5:328  6-10:198  11-20:136  21-40:115  40+:63
  out-degree:  0:12  1-2:260  3-5:308  6-10:167  11-20:124  21-40:34   40+:5
   in-degree:  0:46  1-2:428  3-5:171  6-10:96   11-20:93   21-40:62   40+:14

On the Q3 thresholds (@GioiaZheng): the cascade shape is right, and a mature vault sits comfortably above every global guard — edges/node 6.44 >> 1.0, largest weak component 99.9% >> 50%, well past min_pages/min_edges. So the global guards correctly read "graph-first" at maturity; they're not absurd.

The more interesting signal is the tail: even at 910 pages and mean degree ~13, there's a real low-connectivity fringe — 69 nodes at total-degree 1–2, 46 with in-degree 0. That's the empirical case for keeping the per-seed seed_degree >= 1 / isolated-seed check in the cascade even when the vault is globally dense. The global guard says "this vault is mature"; the per-seed guard still has to catch the individual fresh or peripheral page whose neighborhood PPR can't see yet. Both layers earn their place.

One note for the hub-retirement side (v1.24.0, no rush): avg local clustering is 0.537 vault-wide, which gives the clustering-coefficient retirement signal a realistic baseline — hubs sit well below that (their children don't interlink), so there's clear separation to work with at this density.

Snapshot only; happy to regenerate against a future build if the graph definition shifts — e.g., once PPR settles the canonical node identity for the source/entity duality.

[green-dalii]

v1.23.0 implementation kickoff — 2026-06-27

Picking up this thread for real. Thank you all for the design work — the consensus map is clean and I'm working from it.

What's decided (recap from thread)

• Monte Carlo PPR (Fogaras 2005), not power-iteration — O(K×L) per query, embarrassingly parallel.
• Zero-LLM Tier B — core/section-extractor.ts parses ## Description / ## Definition from existing page bodies.
• Cold-start cascade per @GioiaZheng: lex-only below thresholds, lex-seeded PPR in the middle, graph-first PPR when mature.
• Hub retirement (clustering) — DEFERRED to v1.24.0. @DocTpoint, you said "I already have a reference implementation. Happy to own that as the separate commit" — taking you up on that.
• Embeddings — DEFERRED to v1.25.0+ as opt-in enrichment layer. Anthropic still lacks an embeddings endpoint, others differ in contract.
• Tier C/D — Tier C cancelled (PPR multi-seed replaces it); Tier D deferred to v1.25.0+ (function-calling matrix not stable enough).

Implementation priority

Full table in ~/.claude/projects/.../memory/project_v1.23.0_implementation_plan.md and ROADMAP.md §v1.23.0 Implementation Priority. Quick view:

• P0-1 (blocker): CC0 synthetic 50-page eval fixture at src/__tests__/fixtures/wikis/sample-50page/. PPR quality gate, threshold validation. The 910-page topology aggregate @DocTpoint shared (mean degree 12.88, edges/node 6.44, largest component 99.9%) only sanity-checks global guards; per-seed seed_degree >= 1 thresholds need a fixture to tune. 1 day.
• P0-2 (blocker): Eval script — lex-only vs lex-seeded-PPR vs graph-first-PPR recall@k. 0.5 day.
• P1-1 (independently startable now): core/section-extractor.ts (~30 LOC, 0 LLM, 0 graph dependency, 5+ tests). Fastest ROI — Tier B ships as soon as this lands. 1 day.
• P1-2 (after P0-1): core/monte-carlo-ppr.ts (~30 LOC). Core primitive.
• P1-3..P1-7: hub detection / Web Worker / Query integration / Lint integration / hybrid guard. Total P1: ~5 days.
• P2-1 (welcome note): editable markdown on first launch, per @DocTpoint's founding-note idea. 1 day.
• P2-2 (settings): expose cold-start thresholds so users can tune.
• P2-3 (eval report → ROADMAP): make the baseline permanent as v1.23.0 acceptance gate.

Total: ~7-8 days (5 for P1 + 2-3 for P0/P2 overhead that the consensus raised vs the original ~5-day estimate).

Workflow

Per the project's TDD standard, every module goes failing-test → minimal implementation → refactor → Gate 1 verification. After each phase completes, I'll cut a debug build and pause for manual review before starting the next phase. P1-1 starts today; P0-1 fixture construction starts immediately in parallel.

Open questions (in order of how much they block the first phase)

1. @GioiaZheng — the min_pages=30 / min_edges=30 / edges/pages >= 1.0 / seed_degree >= 1 / largest_weak_component > 50% cascade is the starting point. Are these the right numbers to tune from, or would you rather we start lower and let the fixture tell us? They won't be the shipped defaults either way — fixture says.
2. @DocTpoint — for the welcome note (P2-1), do you want to own that as a separate PR? Same pattern as the clustering retirement. The risk is scope creep (P2-1 touches UI, settings, i18n), so a focused commit might be cleaner. No pressure either way.
3. @alfred1137 — PPR may obviate index.md entirely (graph IS the index). Should we deprecate the index file in v1.23.0 or keep it as a flat fallback for users who don't have a graph? This affects Enhancement: Obsidian Bases for wiki index management #184 scope. Not blocking P1-1 / P0-1.
4. @Linate — the founding note's tag vocabulary seeds PPR basins (different domains → distinct basins → graph is separable). Is there a reconciliation story when new-ingest tags don't match the founding note? Affects schema work, not v1.23.0 directly.

Privacy boundary (per @DocTpoint's call-out)

The public repo gets a synthetic or CC-licensed fixture only — no real vault content, ever. The build script for the synthetic fixture will be part of P0-1. The 910-page topology aggregate @DocTpoint shared stays as the only "real-vault signal" in the design thread.

I'll post phase-by-phase updates here as each module lands, with the recall@k vs lex-only numbers from P0-2 once the fixture is in place.

[alfred1137]

@green-dalii

Open questions (in order of how much they block the first phase)

3. @alfred1137 — PPR may obviate index.md entirely (graph IS the index). Should we deprecate the index file in v1.23.0 or keep it as a flat fallback for users who don't have a graph? This affects Enhancement: Obsidian Bases for wiki index management #184 scope. Not blocking P1-1 / P0-1.

I would say it seem reasonable to prioritize implementing the PPR if it is robust enough to render index management obsolete. Frankly, I cannot really comprehend the conversation so far.

That being said, I am in the opinion that the proposal in #184 can still serve some purposes as "Human-readable UI layer."

The PPR engine solves the problem of automated machine retrieval, while Obsidian Bases index can additionally improve human navigation and content management. When both are implemented, they are addressing different issues from my point of view.

Because both systems rely on identical YAML frontmatter, the Bases implementation should not be major technical debt.

[DocTpoint]

@green-dalii — yes on Q2, happy to own it. One seam adjustment and one addition.

Scope I'd take: the whole mechanism, EN-only. Welcome-note markdown + the field→controlled-vocabulary seeding contract + first-launch wiring (the log.md startup-phase is a clean template) + the opt-in settings flag + tests. What I'd leave to you is i18n across the 10 locales — that's your parity-test harness and translation process, and honestly I can't QC the ja/ko/zh/zh-hant strings, so a contributor hand-rolling them would be worse than your pipeline backfilling them. Standard OSS split: I ship the mechanism + EN strings, your i18n process fills the locales. Keeps it one self-contained PR, same shape as #200.

The load-bearing guardrail, in tests not just prose: the note seeds the tag prior and never materializes domain pages. That's the exact line #197 just closed, so the regression cases fail if a filled-in field ever fabricates a page.

An addition — a second cold-start entry point the existing tagVocabularyMode field already enables. When tagVocabularyMode: 'custom', the user has already declared their domains; the welcome-note's job is half-done before it runs. We can scaffold cold-start anchors directly from that vocabulary — one honest empty seed-node (generation_complete: false, no LLM body) per domain, crystallizing into a hub only as real ingests point at it.

The selection rule is the load-bearing one from the 5-seed onboarding upthread — not all tags, but ≥5 chosen for maximum spread (floor, not cap: 8 distinct domains → 8 anchors). Max-spread self-filters the qualifier tags — a second-position taxonomy qualifier sits near its domain, so it's never picked — which removes the only knob that would otherwise need a manual "is this a domain?" decision.

On how spread is measured at cold start (no graph yet, embeddings deferred): it's a small graceful-degradation cascade, not one metric — vocabulary structure proposes → an LLM completion call ranks spread for a flat custom vocab → optionally a light ingest-folder analysis grounds it in real notes — user confirms at every rung. Worth stressing: a completion call is not an embedding call. Completion is already a hard requirement (ingest/lint/query), so this adds zero new provider dependency — it doesn't reopen the embedding-contract question, and as a one-time user-confirmed seeding step it doesn't touch order-invariance. I'd ship one rung for v1 and leave the rest as named design space.

I'd keep this anchor path as a fast-follow PR to the welcome note, not folded in — same pipeline (vocabulary → spread anchors → crystallizing hubs), second entry point — so the first PR stays focused.

Scheduling: I'm back at a real machine now, so last week's async caveat is lifted — not a blocker either way.

[green-dalii]

A substantive progress report on the v1.23.0 branch.

Completed (Phase 5.1.5 — UX Onboarding + Multi-File Ingest)

• ✅ Three-tier first-run Welcome note (Tier A empty vault / Tier B existing vault / Tier C upgrade). D8 dynamically translated (1 English template → user's wiki language at write time via LLM). A real Obsidian markdown file created on first launch, self-explanatory without needing the README.
• ✅ Feature: in-place batch ingest queue for the Ingest button — avoid moving source notes into a staging folder #130 Multi-File Suggest modal — full in-place batch ingest queue with recursive folder tree, live right-pane progress, per-file cancel. Source notes stay in place (provenance preserved).
• ✅ IngestQueue (pub/sub store) — single source of truth for in-session ingest lifecycle. 25 tests.
• ✅ i18n across all 10 locales — welcome note, modal UI, status strings.
• ✅ PPR-adjacent modules already in the repo: core/monte-carlo-ppr.ts (MC-PPR engine), core/ppr-cascade.ts (hybrid cold-start cascade), core/section-extractor.ts (zero-LLM Tier B).

On the Welcome note vs the design discussion

The Welcome note we shipped is a pure reading guide — install verification, how-to-use, wiki structure explanation, Quick start. No editable domain-declaration fields, no vocabulary-seeding flow, no injected tag vocabulary from Welcome content. This was a deliberate decision.

Two reasons this divergence is the right call:

1. UX reason — forcing the user to define boundaries and scope upfront is unfriendly and inflexible. A first-run form that asks "what domains does your wiki cover?" or "pick 5 spread seeds" assumes the user already knows what they'll be ingesting. In practice, most users discover their vault shape while using the plugin. The Feature: in-place batch ingest queue for the Ingest button — avoid moving source notes into a staging folder #130 multi-file queue lets them start small and grow naturally.

2. PPR doesn't need it. PPR works on whatever graph exists. If the user ingests 3 cardiology notes, those notes' [[wiki-link]] network IS the topology. The hybrid cascade (ppr-cascade.ts) handles the transition from sparse to dense automatically — lex → lex-seeded-ppr → graph-first-ppr — with no user configuration.

On Embeddings

Evaluated from first principles + actual code paths, and I've decided to close the embeddings path entirely, not defer it.

• New pages are already linked by the LLM at ingest time — the link graph is seeded during extraction. The "unlinked page" scenario PPR is blind to doesn't exist in steady-state.
• Cold start is covered by the hybrid cascade's lex-seeded arm.
• Embedding index maintenance (12 MB JSON for 2000 pages, versioning, model-change invalidation, in-memory) contradicts the Karpathy philosophy of knowledge compounding through links, not brittle hidden indexes.
• Cross-lingual is a niche-in-a-niche; single-language vaults are the overwhelming use case.

Decision: No embedding layer. Not for v1.23.0, not for v1.25.0.

P1 (Graph Engine) — next on the branch

The three core modules are already in the repo. Remaining work:

1. Wire ppr-cascade.ts into query-engine.ts.
2. Wire section-extractor into the query context builder.
3. Web Worker wrapper for off-main-thread PPR.

Branch visibility

The feat/v1.23.0-graph-engine-kickoff branch will be pushed to GitHub remote soon for source-level review and testing. Most of the discussion in this thread has been design-only; now there's actual code to look at. Feedback on specific modules is very welcome — the open questions from earlier rounds (e.g., exact cold-start thresholds vs the synthetic fixture) will continue to be discussed here as development progresses.

Look-ahead

@DocTpoint — the clustering-retirement reference implementation is still welcome. A self-contained PR covering core/hub-retirement.ts with the topology aggregates you shared (0.537 avg local clustering at 910 pages) would be a clean v1.24.0 deliverable.

@GioiaZheng — your threshold cascade (min_pages=30 / edges/pages>=1.0 / seed_degree>=1) is the starting point for ppr-cascade.ts. I'll tune against the synthetic fixture.

This thread remains open — if anyone has a concrete case where PPR + hybrid cascade is insufficient, or a better approach to the cold-start question, I'd like to hear it. The project aims to get this right, not just ship it.

[DocTpoint]

No quibble that the cascade runs at cold start — but worth being precise about which arm does the work. With a handful of edges, MC-PPR's per-node estimate is just Binomial(K, p)/K — i.e. the MLE — and for the small-p tail its relative error is ~1/√(Kp). That's the lex arm answering; PPR mostly adds variance until the graph is dense.

Mild irony worth a line: Fogaras MC earns its keep on large graphs — the sparse cold-start regime is exactly where exact power-iteration PPR is both cheaper and variance-free. "The graph is sparse" reads as an argument for exact PPR there, not against helping it separate faster. Not blocking anything — just noting which knob is load-bearing.

[green-dalii]

@DocTpoint — really appreciate both comments. Each one added clarity.

On the vocabulary→spread anchors idea:

The mechanism is well thought through, and the seam you've drawn (you ship EN + mechanism, we backfill i18n) is exactly the right split for this project. I'd like to pick this up as a v1.24.0 discussion rather than a fast-follow to the current sprint — the remaining P1 work (PPR→query integration, lint hub-link strip) needs to land before the graph engine is usable end-to-end, and the cold-start gap you're addressing is already covered by the #130 queue + PPR cascade (lex arm answers until the graph is dense enough). Your proposal adds headroom but the current path doesn't regress without it — so I want to give it the attention it deserves rather than rushing it into an already-packed sprint.

I agree that tagVocabularyMode: 'custom' is the natural trigger, and the max-spread selection rule (≥5 seeds, self-filters qualifiers) is the right design. I'll be thinking about this more as v1.23.0 wraps up — and if you happen to gather more observational data from your vault (e.g. how vocabulary diversity correlates with cold-start time-to-maturity), I'd love to see it. That kind of real-world signal would be invaluable for tuning.

On MC-PPR variance at cold start:

This is a nuance I should have included in my own write-up. You're absolutely right: Fogaras MC earns its keep on large graphs, and the sparse regime is exactly where exact power-iteration PPR is cheaper and variance-free. The cascade already routes sparse graphs to the lex arm, so the practical impact is zero for current users — but I appreciate you taking the time to spell it out anyway. It's a genuinely sharp observation, and it confirms the cascade architecture is on solid ground.

I've noted the MC-vs-exact-PPR question for the synthetic fixture eval — if the results show variance issues at moderate densities, switching to exact PPR in that mid-range band is a one-line routing change in pprCascade. Your breadth across both the applied and theoretical sides of graph algorithms is something I don't take for granted in this project — thanks for bringing it here.

The thread stays open. If anything else comes to mind, I'd love to hear it.

[DocTpoint]

Thanks @green-dalii — v1.24.0 for the anchor path works for me; no reason to crowd the current sprint when the cascade doesn't regress without it.

On the diversity→time-to-maturity data: my current vault is well past cold start, so I can't reconstruct that curve from it retrospectively. But I'm setting up an instrumented fresh run precisely to get it — a separate vault seeded with ≥5 max-spread anchors, ingesting a controlled subset and snapshotting topology (modularity over the domain-tag partition, core size, clustering) after each batch. That measures the seeding→separation relationship directly, and it composes with the MC-vs-exact eval you flagged. I'll share aggregates once I have a clean series.

Near-term I'll get the hub-retirement signal to you as the self-contained core/hub-retirement.ts we discussed — that one's ready to port.

[GioiaZheng]

Thanks for the implementation update. The current ppr-cascade direction matches how I'd frame the thresholds.

On the numbers: I would treat min_pages=30 / min_edges=30 / edges/pages >= 1.0 / seed_degree >= 1 / LWC > 50% as tuning priors, not defaults to defend upfront. Start conservative, then let the fixture + eval move them.

I would split the guard into two layers:

• vault maturity: min_pages, min_edges, edges/pages, largest weak component. This decides whether graph retrieval is eligible at all.
• seed readiness: seed_degree, plus maybe reachable neighborhood size or top-k reachable mass. This decides whether a specific query should go graph-first, lex-seeded PPR, or lex-only.

The 910-page aggregate is useful evidence that the mature-vault arm is sane, but I would not use it to justify cold-start thresholds by itself. It says the global graph has enough structure; it does not say every query seed has enough local support. That is exactly why the per-seed guard matters.

On MC vs exact PPR, I agree with @DocTpoint's nuance. MC is attractive once the graph is large enough that local sampling wins, but sparse/mid-density graphs may be better served by exact PPR or by staying lex-seeded until variance stops moving top-k. I would let the fixture compare three routes: lex-only, lex-seeded exact PPR, and lex-seeded/graph-first MC-PPR. If exact wins in the middle band, routing through it is just an implementation detail, not an architecture reversal.

On embeddings: no objection to keeping them out of v1.23.0. The only line I would preserve in the design notes is a future opt-in enrichment escape hatch if real user fixtures show a relation class that link extraction consistently misses. That keeps current scope clean without overclaiming that links cover every semantic case.

[DocTpoint]

Re: observation data on anchor/vocabulary diversity ↔ cold-start maturity — I ran a small controlled A/B and the result lands squarely on your first-principles side, so it seems worth sharing.

Setup. One 50-note corpus spanning 6 domains (microbiome, immunology, mitochondria, metabolism, nutrition, psychology), identical model and settings (local Gemma, temp 0.15), the only variable being ingest order:

• Arm S (seeded): the 6 domain anchors first, ingested one at a time (each a lean ~20-line, link-free note, so a seed contributes few edges), with a snapshot after each; then the remaining 44 notes as a batch.
• Arm U (batch): all 50 notes as a single unordered folder ingest.

I measured link-intrinsic topology (Louvain modularity over the wiki-link graph, k-core / 3-core fraction, average clustering, connected components) — deliberately not the tag partition, since tags don't drive retrieval.

Result (end state, same corpus):

metric	S (seeded)	U (batch)
nodes	496	521
3-core fraction	76.2%	77.7%
Louvain Q	0.669	0.672
avg clustering	0.565	0.565
components	1 (100%)	1 (100%)

Statistically indistinguishable on every macro-structural metric. During the seeding phase the domains do nucleate as separate components and then merge along genuine shared-concept bridges (e.g. cytokines pulling the molecular domains together; psychology stays isolated until the body notes bridge it via stress/HPA concepts) — so you can watch the cold-start dynamics — but the mature topology converges to the same place regardless of order.

Takeaway. For this corpus, deliberate spread-seeding buys a transiently cleaner early structure but no lasting topological advantage: the link graph the cascade needs emerges from the content, not the ingest sequence. That matches your read that anchors are an optimization rather than a necessity. The one consistent (but n=1) difference was redundancy, not structure — the batch arm ended with slightly more duplication (521 vs 496 nodes; 34 vs 27 high-confidence dedup candidates), i.e. seeding gave later notes canonical merge targets. Marginal, and below the noise floor of single runs.

Caveats: n=1 per arm (LLM extraction is stochastic — hence the 496 vs 521 node gap; the macro-metrics converging despite different node sets is itself the robustness signal), single corpus, single local model.

Happy to share the snapshot script (pure-stdlib Louvain + k-core, deterministic) and the per-step CSV if either is useful for the fixture-eval you mentioned for the PPR routing.

[DocTpoint]

Follow-up observation that bears on the "structure emerges from content" framing — this time about the quality of the link graph the cascade consumes, not just its order-invariance.

I'd been assuming our vault's dense core (≈76% 3-core) was partly an LLM artifact — specific pages lazily linking to prominent hubs, inflating interconnection. I measured it three ways and the assumption was wrong on every axis:

1. Is the core hub-scaffolded? (structural, link-graph only)
Removing the top-30 highest-degree hubs entirely drops the 3-core from 75.9% to only 63.8%. Of the core-internal edges, just 9% touch a top-5 hub, 28% a top-20 hub — >70% of the dense core is peer-to-peer. The core does not hang off hubs.

2. Are the hub-links themselves lazy? (bge-m3 embeddings, distinctness = cos(P,hub) − centrality(hub))
83% of evaluable hub-links (255/308) clear a conservative distinctness floor — majority are genuine, with only a ~17% generic tail.

3. Do peer↔peer edges discriminate semantically? (AUROC of edge-cos vs random-pair-cos)
AUROC 0.897 (n=1195 peer edges vs 8000 random pairs; edge-cos median 0.611 vs 0.458 random). A real edge is more semantically similar than a random pair ~90% of the time. Median link target sits at rank ~23 of 673 candidates — upper few percent of the source's nearest neighbors.

Takeaway: the dense core is genuine content density, not a linking artifact. For a graph-based retrieval engine this is the encouraging read — the link substrate the PPR cascade walks is semantically sound (edges track relatedness, AUROC 0.90), so PPR is propagating over real signal rather than noise. It also means no structural intervention (active hub-suppression or anchor-engineering during ingest) is warranted — there's no artifact to correct, which lines up with your first-principles call that the structure should be left to emerge.

Caveats: single corpus (medical/biology, genuinely densely connected), single local embedding model (bge-m3), distinctness/centrality are relative measures that drift as the corpus grows. Scripts (pure-stdlib graph + the embedding AUROC harness) available if useful for the fixture-eval.

[green-dalii]

@DocTpoint
Appreciate the two recent posts — the cold-start experiment and the link-graph analysis. Both are directly relevant to PPR tuning I just finished, so sharing the numbers here.

Our end: leave-one-out eval on a 2142-page real vault (3473 vertices, 12158 edges), 6 param combos tested:

damping	walks	walkLen	lex R@5	cascade R@5	cascade R@10
0.15	1000	20	10.5%	21.5%	37.2%
0.10	1000	20	10.5%	22.3%	37.9%
0.10	3000	20	10.5%	22.3%	37.8%
0.05	3000	20	10.5%	23.8%	39.6%
0.05	3000	40	10.5%	23.5%	38.2%
0.05	5000	40	10.5%	23.7%	38.5%

Winner: damping=0.05, walks=3000, walkLen=20. PPR improves lex by 2.3×.

Where it lines up with your data:

• Your AUROC 0.897 (edges track relatedness ≈90% of the time) explains why a simple PPR cascade works this well — it's walking real signal.
• Your 3-core finding (>70% peer-to-peer) means no hub-suppression filter needed. Cascade stays lean.
• The cold-start result (seeded vs batch converge to the same topology) confirms structure emerges from content, not ordering.

Also landed: PR #215 (@DocTpoint's hub-retirement module). Judges hub-tag lifecycle, not hub-suppression. Zero callers in current release — headed for future lint integration.

Two questions your posts raise:

1. Your corpus is medical/biology; ours looks science-heavy too (541:1571 entity/concept ratio). Would a general-interest vault with lower link density get the same PPR lift? The synthetic fixture suggests "yes, but the gap shrinks".
2. Your AUROC 0.897 is on bge-m3. Should we add a "PPR vs nearest-neighbor" baseline to separate "graph is well-structured" from "PPR is specifically good at it"?

The full eval table is up at src/__tests__/fixtures/wikis/sample-50page/REAL_VAULT_EVAL.md on the refactor/v1.23.0-ai-sdk-migration branch. Open for review.

[DocTpoint]

@green-dalii — ran the nearest-neighbor baseline you asked for (Q2), on your own sample-50page fixture so lex/cascade reproduce your numbers exactly and only the new arm is mine.

Setup. Same 53 pages, same 10 ground-truth queries, same R@k. Three arms:

• lex / cascade — your eval-cascade.ts unchanged (reproduced: lex 13.3% R@5, cascade 27.1% / 37.8%).
• knn — bge-m3 embedding of the query string, cosine-ranked against bge-m3 embeddings of page text. No graph, no lex.

arm	R@5	R@10
lex (keyword)	13.3%	13.3%
knn (bge-m3, no graph)	24.1%	36.4%
cascade (graph)	27.1%	37.8%

Read. A pure semantic retriever with zero graph lands within ~1–3 pts of the cascade (R@10 36.4 vs 37.8); both ≈2× the keyword baseline. So most of the cascade's lift is semantic-over-keyword, not graph-over-semantic — the lex baseline understates what a non-graph retriever gets. That's exactly the separation Q2 was after: "graph is well-structured" and "PPR is specifically good at it" turn out to be largely the same fact (the structure tracks semantics — consistent with the AUROC 0.90), so PPR ≈ embedding-kNN as retrievers.

Where they diverge — the interesting part, they're complementary by query type:

• AF rate control: knn 0.43/0.71 beats cascade 0.14/0.29 — embeddings resolve the synonym.
• Pinewood Heart Center: knn 0.00, cascade 0.20/0.40 — proper-noun/structural query, semantics whiff, the graph walks the source cluster.

That divergence is the precise relation-class for @GioiaZheng's opt-in enrichment escape-hatch: embeddings catch conceptual synonyms link-extraction misses; the graph catches structural/proper-noun cohesion embeddings miss.

So the honest case for the graph engine isn't retrieval quality — it's cost. It delivers embedding-grade R@k with no embedding model, offline, over links that exist anyway. Strong argument, just a different one than "2.3× over lex" implies. If you keep that table around, I'd report cascade vs knn next to cascade vs lex — it's the control that keeps the cascade's value from being over-stated.

Complementary regime (second eval). On our real vault (902 entity/concept pages, 2669 edges) I ran the inverse task — leave-one-out link prediction, query = the full source page, recover one held-out neighbor:

arm	R@10	MRR
lex (tf-idf)	0.364	0.171
knn (bge-m3)	0.354	0.172
ppr (single-source)	0.265	0.088

Here knn ≈ lex ≫ single-source PPR. When the query is rich (a whole page of context), content similarity already captures relatedness and graph propagation from one node trails it. Combined with the fixture: the graph earns its lift specifically in the impoverished-query regime — a short keyword that names a hub but not the cluster. That's the cold-start case, and a clean characterization of when the cascade helps.

Re your two follow-ups: (1) lower-density/general-interest vault — agree it shrinks, but the governing variable is edge-density × edge-AUROC (the PPR retrieval ceiling), independent of the MC-vs-exact estimator question. (2) Yes, add the NN baseline — it's the one control that separates "well-structured" from "PPR-specifically-good," and on this fixture they're the same thing.

Caveats: fixture n=10 queries, single corpus each (both medical/biology, English), single embedding model (bge-m3); knn is query→page while the cascade is the production path. Scripts (kNN fixture arm + LOO harness, pure-stdlib + LM-Studio embeddings) available if useful for the fixture-eval.

[green-dalii]

@DocTpoint — thanks for turning the Q2 baseline around so fast, and especially for grounding it in the same fixture so the comparison is apples-to-apples. Cascade vs knn vs lex on identical queries is the control I was missing; the per-query divergence table is exactly the granularity I needed.

Where this lands for v1.23.0:

1. P2-3 eval gate now reports cascade vs knn alongside cascade vs lex. Your numbers go in as the control baseline. Attribution in the eval report will reflect your framing: "Cascade's 2.3× over lex is mostly semantic-over-keyword, not graph-over-semantic. Cascade's honest value: embedding-grade R@k at zero embedding cost, offline, over links that exist anyway." That re-frames the win correctly — we're not overclaiming retrieval quality, we're claiming zero-cost retrieval that's good enough.

2. No opt-in embedding path in v1.24.0+ roadmap. 3pp gap (cascade 27.1% vs knn 24.1%) doesn't justify the 9-provider embedding matrix, and adding a per-query classifier that decides knn vs cascade arm would couple an architectural decision to query-class heuristics we don't have data for. Sticking with the [[wiki-link]] graph as the free semantic layer (consistent with the v1.22.0 Architecture: a shared embedding "structural awareness" layer behind #117, #157, dedup, and #164-grounding #175 rejection).

For your kind offer to run additional evals on your private vault — yes, please, whenever you have time. Not urgent; v1.24.0+ planning, not the v1.23.0 release on 2026-07-02.

Three things that would be useful (each independent, take whatever fits your bandwidth):

• PPR eval on your 902-page vault with broader param sweeps. Our T3 (damping=0.05, numWalks=3000, walkLength=20) was the winner on a 2142-page corpus, but that's one vault — it could easily be a local optimum. If you have cycles, sweeping damping across 0.02 / 0.05 / 0.10 / 0.15 and walks across 1000 / 3000 / 5000 on your 902-page vault would tell us whether the optimum generalizes or is corpus-specific. No pressure to use the same params — explore freely.
• General-interest / lower-link-density vault eval. Your earlier question about whether the lift generalizes. Lower edge density × edge-AUROC is the governing variable per your own framing; would love to see the actual numbers on a less densely-linked corpus.
• Cold-start regime exploration. Your seeded vs batch experiment showed topology converges regardless of order — interesting. If anything else catches your eye in that regime (e.g. whether a different seed-set construction matters, or whether the divergence pattern knn-wins-synonyms / cascade-wins-structural shifts at low edge counts), happy to discuss. No specific architectural proposal here from my side; just keeping the door open for any pattern you'd find useful to surface.

Whenever you have spare cycles — and only if it's fun. The scripts and harness you've already shared (kNN fixture arm + LOO link-prediction harness, the AUROC distinctness harness) are extremely valuable; if you can publish them or share a link to a branch, I'll fold them into the fixture-eval repo alongside the cascade harness for future contributors.

Special thanks for the quality of this thread. Three rounds of evidence — the link-graph quality analysis (AUROC 0.897, 3-core >70% peer-to-peer), the seeded-vs-batch cold-start experiment, and now the knn baseline — each one independent, each one grounded in the same fixture methodology. That's exactly the kind of contribution that lets us ship v1.23.0 with confidence rather than hand-waving. Genuinely appreciated.

[green-dalii]

Closing this thread now that v1.23.0 shipped — the PPR cascade architecture you proposed is live, and the data points from this discussion directly shaped the implementation. Thank you all for the depth; this is the kind of design work that makes a project better.

What shipped in v1.23.0 (closed by this release):

• Graph Engine as a single primitive powering hub-detection (Feature: Graph-based hub detection for automatic domain tag inference #117), link-distinctiveness (Proposal: replace hub-tag link stripping with embedding-distinctiveness (fixes over-stripping of legitimate broad→hub links) #157), Query retrieval (Architecture: Personalized PageRank (PPR) graph engine — v1.23.0 direction (consolidates #117, #157, #175) #198), and the dead-link hub check — core/monte-carlo-ppr.ts (Fogaras 2005 MC-PPR, K random walks at O(K×L) cost independent of |V|), core/ppr-cascade.ts (3-tier cascade with hybrid guard), core/section-extractor.ts (zero-LLM Tier B). Real-vault tuning on 2142 pages: damping=0.05, numWalks=3000, walkLength=20 improved cascade R@5 from 21.5% → 23.8% (+11% relative).
• Cold-start cascade with the two-path framework @GioiaZheng proposed: empty/small vaults get pure lex, mid-density gets lex-seeded PPR, mature vaults get graph-first PPR. The min_pages / min_edges / edges/pages / LWC vault-maturity guards + seed_degree / reachable-neighborhood seed-readiness guards are all in core/ppr-cascade.ts as pure-function thresholds.
• Tier B redesign (zero LLM): parses ## Description / ## Definition from existing page bodies — no schema migration, no compatibility issue with old wikis.
• Hub-retirement crystallization signal (@DocTpoint's PR Add hub-retirement crystallization signal (core/hub-retirement.ts) #215) merged as a self-contained pure module. Wired for lint integration in v1.24.0.

Empirical evidence that informed the implementation:

• Your knn baseline (cascade R@5 27.1% vs knn 24.1% = 3pp gap on the sample-50page fixture) reframed how I described the win in the README — "embedding-grade R@k at zero embedding cost, offline, over links that exist anyway." Most of cascade's lift is semantic-over-keyword, not graph-over-semantic.
• Your 910-page topology aggregate (mean degree 12.88, edges/node 6.44, LWC 99.9%, 3-core 76%) was used to sanity-check the global guards. No hub-suppression filter is needed — the dense core is peer-to-peer, not hub-scaffolded.
• Your peer-link quality analysis (AUROC 0.897, n=1195 peer edges vs 8000 random pairs; hub-links 83% clear the distinctness floor) explained why a simple PPR cascade works: it's walking real signal.
• Your cold-start A/B experiment (seeded vs batch ingest converge to identical topology on macro-structural metrics) confirmed structure emerges from content, not ordering — the cascade doesn't regress without the guided-seeding path, which is why we deferred that to v1.24.0 instead of crowding the current sprint.

Decisions that diverged from specific proposals:

• Embeddings → opt-in in v1.25.0+ (deferred from v1.24.0). The 3pp gap doesn't justify the 9-provider embedding matrix. @GioiaZheng, your "embeddings as opt-in enrichment layer behind a capability probe" framing is the right long-term shape, but it needs more provider-support work than a single sprint could carry. Deferring doesn't reject the framing — it just stages it after the core PPR work is battle-tested.
• MC vs exact PPR in the cascade: I went with MC as the default path for simplicity (one knob to tune) and used the min_pages / min_edges thresholds to short-circuit to pure-lex in the sparse regime where exact would be cheaper. Your "let the fixture compare three routes" point is well-taken — I'll add exact-PPR as a routing option in v1.24.0 if the MC variance causes top-k stability issues in real-world testing.
• Welcome note scope: shipped as a pure reading guide (install verification, how-to-use, structure explanation) — no editable domain-declaration fields, no injected tag vocabulary from Welcome content. Forcing scope-boundary declaration upfront felt unfriendly and inflexible. The intent is real (Phase 5.1.5), the form isn't what we initially sketched in this thread.
• @alfred1137, your Enhancement: Obsidian Bases for wiki index management #184 Obsidian Bases point stands — Bases as human UI layer and PPR as machine retrieval layer are complementary. The two can ship independently; Bases is on the v1.24.0+ roadmap alongside Add PDF support #218 (PDF source ingest).

What's next (v1.24.0+):

• Add PDF support #218 PDF source ingest (design in Discussion PDF Source Ingest for v1.24.0 — open design discussion #222)
• Source-revision awareness for merge: distinguishing self-updates from cross-source conflicts #220 source-revision awareness (Tiers 0-1 — fingerprint function redesign + supersedes: frontmatter flag)
• Hub-retirement lint wire-up (currently 0 callers; the signal exists, the lint integration is next)
• P2-2 cold-start guided seeding (your anchor proposal, @DocTpoint) — instrumented fresh-run data from your A/B experiment would be the signal to tune min_pages / seed-spread thresholds
• v1.23.2 PATCH: Add setting to suppress persistent progress Notice pills during lint/ingest operations #219 Progress Notice suppression + Add option to scroll to the start of a completed query response #221 Query scroll-to-start (filed by @jameses-cyber)
• v1.23.1 PATCH (already shipped, 2026-07-02): Obsidian review-bot hotfix — strictBindCallApply: true alignment, dead function removal, lockfile regeneration. No user-facing changes from v1.23.0. https://github.com/green-dalii/obsidian-llm-wiki/releases/tag/1.23.1

Special thanks:

• @DocTpoint — hub-retirement signal (PR Add hub-retirement crystallization signal (core/hub-retirement.ts) #215), 910-page topology aggregate, knn baseline, cold-start A/B experiment, peer-link AUROC analysis, max-spread anchor proposal for v1.24.0. The methodological rigor of your evaluations (separate arms, matched fixtures, controls) is what made the PPR-vs-embedding decision defensible.
• @GioiaZheng — PPR-vs-embedding framing correction, two-path cold-start framework, vault-maturity vs seed-readiness threshold architecture, MC-vs-exact PPR nuance, opt-in enrichment layer shape for v1.25.0+.
• @alfred1137 — for confirming Enhancement: Obsidian Bases for wiki index management #184 (Obsidian Bases) complements rather than competes with PPR. The split between human UI layer and machine retrieval layer keeps both discussions clean.

If any of you want to run your own evals on the v1.23.0 cascade or its MC-vs-exact behavior on private vaults, the eval script is at src/__tests__/core/eval-cascade.ts and accepts your own fixtures. Aggregate-only feedback (no content) is the easiest shape to share back, but full eval traces are welcome too.