Skip to content

Knowledge Graph

Jump to bottom
Chris Sweet edited this page Jul 14, 2026 · 2 revisions

type: reference up: "What-is-the-llm-wiki-Template" supports: "Template-Philosophy" related:

Knowledge Graph

The template ships an optional pipeline (scripts/kg/) that turns the wiki's frontmatter and body links into a SPARQL-queryable RDF graph. This page explains it for users who want to understand or query it. Grounded in scripts/kg/README.md, the template wiki's Knowledge-Graph-Pipeline page, and Edge-Types.md.

Why does the template build a knowledge graph from the wiki?

To answer topology questions the prose cannot: "which pages cite finding X?", "what does this experiment depend on, transitively?", "find all pages that criticizes: a parent of Z". The frontmatter typed edges already encode these relationships; the KG pipeline makes them queryable. Without typed edges the graph collapses to a flat mesh of mentions; with them, the same query can ask not just "which pages connect to X" but "which pages support a claim that X". Edge typing is what turns the graph from a navigation aid into a reasoning substrate. See Template-Philosophy on why the template still does not treat the graph as the primary retrieval mechanism.

Do I need to know SPARQL to use the template?

No. The wiki works fully without ever touching the graph: index-first navigation and file reads answer content questions. The KG is opt-in, for topology questions and maintenance queries. When you do query it, the template ships a library of canned SPARQL queries so you can run common questions without writing SPARQL yourself (see below).

How do I rebuild the graph after wiki edits?

Run ./scripts/kg/build-graph.sh. It is idempotent, so re-running is safe. Useful flags: --wiki=PATH for a custom wiki location, --refresh-spec to re-fetch the LA3D ontology/SHACL spec, --stats for extractor coverage, --help for the full list. Rebuild after each ingest, or periodically.

What does scripts/kg/build-graph.sh do?

It is a thin shell entry point over build-graph.py. It walks the wiki, extracts frontmatter and body links (via wiki-to-jsonld.py), produces the graph as JSON-LD and Turtle, materializes derived structure, and runs SHACL validation. The whole pipeline runs in-process Python using rdflib and pyshacl: no Java, no server, no subprocess. It does not abort on validation failures; it writes the report and surfaces a summary line.

What does wiki-to-jsonld.py extract from the wiki?

The frontmatter-plus-body extractor. It reads each page's YAML frontmatter and turns the typed-edge fields (up, extends, supports, criticizes, source, related, and the rest of the LA3D forward predicates) into typed edges, and turns body cross-references ([text](Page-Name)) into mentions edges. It also recognizes the inline Variant 1 annotations and HTML-comment attributes. Pages without frontmatter are still included as untyped nodes, so no page is dropped.

What outputs land in scripts/kg/build/ and what is each for?

The build/ directory is gitignored (runtime-only). It contains:

File Contents
graph.jsonld JSON-LD extracted from frontmatter + body links
graph.ttl Turtle translation, with RDF-star weights appended
graph-weights.ttl RDF-star weighted mentions edges from the extractor
graph-full.ttl graph.ttl plus materialized inverses, hub flags, area inheritance
validation-report.ttl the SHACL conformance report

graph-full.ttl is the one to load for querying; it carries the materialized structure.

What is RDF Turtle, and why does the template use it?

RDF Turtle (.ttl) is a compact, human-readable text serialization of an RDF graph (subject-predicate-object triples). The template uses it because it is the standard, tool-agnostic interchange format for RDF: rdflib reads and writes it natively, it loads into any triplestore (including Fuseki), and it is diffable in git. The pipeline also emits JSON-LD (the JSON serialization of the same graph) as the extractor's native output. See Concepts-Glossary for RDF Turtle, RDF-star, and JSON-LD.

What is materialized (inverses, area inheritance, hub flags) and why?

Materialization means computing derived triples once at build time so queries do not have to. Three kinds, run as SPARQL CONSTRUCT queries against the loaded graph:

  • Inverse edges: for each forward edge the pipeline emits its inverse (extends → extendedBy, supports → supportedBy, ...), so a query can traverse a typed edge in either direction. This is why agents never author inverse predicates by hand: the build does it.
  • Area inheritance: a page inherits area/domain context from its hierarchy, so a query scoped to an area picks up descendants.
  • Hub flags: pages with many inbound links are flagged, so "find the hubs" is a simple query.

What SPARQL queries ship with the template?

scripts/kg/sparql/ carries a curated set (11 in this bundle), including: ancestors-of, children-of, extension-chains, graph-stats, hub-notes, note-neighborhood, notes-by-tag, notes-by-type, orphan-notes, search-by-title, and supports-criticizes. These cover the common topology and maintenance questions (hub detection, orphan detection, edge-typed traversal, neighborhood expansion) without your writing SPARQL.

When would I write my own SPARQL query?

When your topology question is not covered by the canned set: a domain-specific traversal, a multi-hop pattern particular to your wiki's edge vocabulary, or a custom health check. The canned queries are a starting library, not a ceiling. Reach for a custom query only for topology ("what connects to what"); for content ("what does this page say") read the file directly. That topology-vs-content split is the template's core retrieval guidance.

What is SHACL validation and what does a failure mean?

SHACL is a constraint language for RDF: shapes describe what a conformant graph should look like (expected node types, required predicates, domain/range on edges). pyshacl.validate() checks the extracted graph against the fetched shapes.ttl and writes validation-report.ttl. A failure (a "shape violation") means a page's frontmatter does not satisfy a shape, e.g. an edge pointing at the wrong kind of target, or a missing required field. The build does not abort on violations; it reports them for you to fix. See Troubleshooting for reading a report.

Why is the pipeline in Python (rdflib + pyshacl) rather than Java (Jena)?

Deliberately, so the whole pipeline is a pip install away with no system-level dependencies: no Java, no Apache Jena, no Fuseki required by default. JSON-LD parsing, Turtle serialization, materialization (SPARQL CONSTRUCT via rdflib.Graph.query()), and SHACL validation all run in-process. This keeps the agent-tool layer that builds on the KG in Python, so calls into the graph are in-process method calls rather than subprocess invocations of a CLI. Dependencies are just Bash 4+, Python 3 with PyYAML, rdflib, and pyshacl.

When would I switch to a Fuseki endpoint?

When you need something the in-process mode cannot give: a live SPARQL endpoint for multiple concurrent clients, agent-write via SPARQL UPDATE, federated queries across several wikis, or a web dashboard. Apache Jena Fuseki can host the produced graph-full.ttl, and rdflib talks to it via SPARQLStore with no change to tool code. This is opt-in; the default is in-process. The template notes that a future cross-session agent-memory feature is expected to depend on the Fuseki path, because cross-session writes need SPARQL UPDATE and an HTTP surface.

A note on distribution

The KG pipeline is present in this bundle under scripts/kg/. In the template's own distribution model it is not yet in the ALWAYS_FILES sync contract, so a plain update-from-template.sh does not automatically push it into every derived project; the likely path is shipping it as an opt-in feature. If your derived project lacks scripts/kg/, that is why. See Extending-the-Template.

See also

Clone this wiki locally