Skip to content

edaphos 1.0.0 — Pillar 1 at paper scale: persistence, RDF export and audit

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 16:18
· 175 commits to main since this release

Summary

First major version: a Knowledge Graph built from tens of thousands
of abstracts is no longer an in-memory dead end. Four additive
capabilities turn Pillar 1 into a research artefact that can be
persisted, federated with external RDF vocabularies, and
interrogated by standard audit primitives.

Persistence

  • causal_kg_save() / causal_kg_load() serialise an
    edaphos_causal_kg through its tidy edge list (not through
    igraph's raw C-level pointer layout), so the resulting .rds
    is portable across igraph versions and byte-reproducible.
    Loading reruns the duplicate-edge merge + cycle check so a saved
    KG round-trips exactly to a freshly built one. The header
    carries a format_version tag, a saved_at timestamp and the
    edaphos / R version metadata.

RDF 1.1 Turtle export

  • causal_kg_to_turtle() emits a W3C-conformant RDF 1.1 Turtle
    document (Beckett et al. 2014). Each edge becomes a reified
    rdf:Statement carrying confidence, evidence, source(s) and
    timestamp, so the full provenance stack survives the round-trip;
    each node is assigned a stable IRI inside a user-controlled
    namespace so the KG federates cleanly with AGROVOC, ENVO and
    other SKOS thesauri via owl:sameAs. The emitter is pure R — no
    RDF library dependency — and the output parses against any
    RDF 1.1-conformant consumer (rdflib, Jena, Oxigraph, Blazegraph,
    GraphDB, Virtuoso).

Paper-scale audit

  • causal_kg_rank_edges(by = c("n_sources", "mean_confidence", "agrovoc_support")) collapses the KG to unique
    (cause, effect) pairs and sorts by a priority list of metrics.
    The single most informative signal for an LLM-extracted KG is
    n_sources — an edge asserted by 50 papers is far more
    trustworthy than one asserted by 1. An optional alignment
    argument attaches agrovoc_cause, agrovoc_effect and
    agrovoc_support columns so the caller can prefer edges whose
    endpoints resolve to community-governed vocabulary.

  • summary.edaphos_causal_kg() is the one-line health check:
    node count, edge count, unique source count, confidence
    quartiles, DAG-ness verdict, and the most prolific source.

Concurrent AGROVOC alignment

  • causal_ontology_agrovoc_align_batch() dispatches N
    single-term SPARQL queries concurrently through
    httr2::req_perform_parallel() with a user-controlled
    max_active, idempotent on-disk cache, and exponential-backoff
    retries. Measured speedup of roughly 5× at max_active = 5
    against agrovoc.fao.org, with warm-cache re-runs ~3700× faster
    than cold.

    A note on the batching strategy: true SPARQL-level batching
    (VALUES + CONTAINS(?label, ?term)) is rejected by AGROVOC's
    production endpoint with a 504 gateway timeout because the
    substring filter cannot short-circuit against a bound term set.
    agrovoc_align_batch() therefore batches at the transport layer
    instead — identical on-wire semantics, only the wall-clock time
    changes.

  • causal_kg_alignment(kg, vocab = "agrovoc", agrovoc_batch = TRUE, agrovoc_max_active = 5L) is the KG-level one-liner that
    routes the batched resolver through the standard alignment
    output.

Documentation

  • Vignette pilar1-causal gains §11 "Paper-scale audit:
    persistence, Turtle, ranked edges" deriving each primitive from
    first principles with reproducible toy-KG examples.
  • README gains a "Paper-scale persistence, Turtle export and
    multi-source audit" subsection with a complete save → Turtle →
    align → rank walk-through.

Quality

  • 62 new tests land in test-causal-v10.R covering the RDS
    round-trip, the Turtle emitter (prefix declarations, reified
    edges, escape rules, empty-KG handling), the ranker
    (multi-key sort, AGROVOC support, top_n), summary() and the
    mocked parallel-dispatch variant.
  • Full causal test suite: 167 / 167 green.
  • R CMD check --as-cran: 0 errors / 0 warnings / 2 harmless
    NOTEs (network timestamp + PySCF tempdir carried over from
    v0.9.0).