Skip to content

Releases: HugoMachadoRodrigues/edaphos

edaphos 1.6.0 — Unified uncertainty across the six pillars

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 14:43

Summary

Every pillar's uncertainty output is now expressible as the same
S3 object — edaphos_posterior — and admits the same
calibration diagnostic, plotting routine and adapter protocol.
Calibrating the six pillars against their natural ground truth now
takes three lines of code.

New core infrastructure

  • edaphos_posterior() — S3 class carrying either a
    (n_samples, query_shape) sample array or a Gaussian
    (mean, sd) summary, with pre-computed quantile fields, an
    optional epistemic / aleatoric decomposition, a method tag
    ("ensemble" | "bootstrap" | "mcdropout" | "bayesian" | "loo_cv" | "analytic" | ...) and a query-type tag
    ("effect" | "map" | "sample" | "feature" | ...).
  • uncertainty_calibrate() — single diagnostic returning CRPS
    (Gini-mean-difference Monte-Carlo formula of Gneiting & Raftery
    2007), PICP and MPIW at each requested nominal level, a
    reliability data frame and the point RMSE.
  • autoplot.edaphos_posterior() — ggplot2 dispatch on
    query_type.
  • uncertainty_plot_reliability() — reliability-diagram plot.
  • as_edaphos_posterior() — S3 generic adapted to every pillar.

Per-pillar adapters

Pillar New API Method
1 causal_effect_posterior(), causal_effect_bootstrap() cluster-block bootstrap (LM) / BART posterior
2 piml_neural_ode_posterior(), piml_bayes_posterior() K-seed deep ensemble / Laplace or MCMC
3 temporal_convlstm_ensemble_fit(), temporal_convlstm_ensemble_rollout(), temporal_convlstm_mcdropout_predict() K-seed ensemble / MC-dropout / Kalman analysis
4 foundation_finetune_ensemble(), foundation_mcdropout_predict() K-seed head ensemble / MC-dropout head
5 active_learning_posterior() QRF quantile grid
6 quantum_krr_posterior() GP-equivalent analytic posterior

New vignette

vignette("uncertainty-unified") — compact end-to-end tour
through the six pillars with the same three-line recipe
(as_edaphos_posterioruncertainty_calibrateautoplot),
the single CRPS / PICP@95 / MPIW@95 / point-RMSE table that is
now the package's headline diagnostic, and a faceted reliability
diagram.

Deliverables

  • R/uncertainty.R — the core infrastructure (constructor,
    calibration, autoplot dispatch, reliability plot, generic adapter).
  • R/{causal,piml,temporal,foundation,active,quantum}_posterior.R
    — one file per pillar with the adapter + pillar-specific posterior
    helpers.
  • tests/testthat/test-*-posterior.R126 new tests across
    seven files.
  • vignettes/uncertainty-unified.Rmd — the calibration-table
    narrative.

Quality

  • R CMD check: 0 errors / 0 warnings / 1 unrelated NOTE
    (future file timestamps — system-clock artefact).
  • Backwards compatible: no existing API was renamed or removed;
    every new entry point is additive.

edaphos 1.5.0 — Pillar 3 on real data + localized stochastic EnKF

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 13:52

Summary

Takes Pillar 3 from the synthetic temporal_synth_soc_cube() toy to
real Cerrado spatio-temporal data: a 10 × 10 × 168-month × 3-channel
cube over a 2° × 2° AoI (Goiás / Minas Gerais triple junction, Jan 2010 –
Dec 2023), and ships a stochastic Ensemble Kalman Filter with
optional Gaspari–Cohn localization for sequential assimilation of
new in-situ observations into a trained ConvLSTM forecast.

New API

  • temporal_kalman_update() — stochastic EnKF (Evensen 1994;
    Burgers, van Leeuwen & Evensen 1998) on a 3-D (N_ens, H, W) or
    4-D (N_ens, H, W, T) forecast ensemble. Returns the posterior
    ensemble, posterior mean / SD maps, per-observation Kalman-gain
    norm, and the innovation vector.
  • Optional localization_radius — Gaspari & Cohn (1999)
    5th-order polynomial taper on the gain, the standard small-ensemble
    fix (Houtekamer & Mitchell 2001) for spurious long-range
    correlations.

The 4D Cerrado cube

  • NDVI — MOD13Q1 250 m 16-day composites, aggregated to monthly
    means (NASA LP DAAC).
  • Precipitation — NASA POWER monthly mean daily precipitation
    (mm/day, MERRA-2 bias-corrected) scaled by days-in-month.
    (Original target was CHIRPS but the CHG data portal was returning
    HTTP 403 for every global_monthly/tifs/ request at the v1.5.0
    freeze; POWER is fully open and comparably accurate over the
    Cerrado.)
  • Air temperature — NASA POWER T2M (MERRA-2 bias-corrected),
    year-specific rather than a static climatology — an upgrade over
    the originally-planned WorldClim 2.1 pack, enabled by POWER
    returning both channels in one REST call per cell.

Rollout + assimilation result

Metric Value
K ensemble members 10
past window (training) Jan 2010 – Dec 2020 (132 mo)
future window (forecast) Jan 2021 – Dec 2023 (36 mo)
target month Dec 2023
in-situ observations assimilated 8
obs noise SD (NDVI z-units) 0.15
localization radius (cells) 2
prior RMSE (NDVI z-units) 0.637
analysis RMSE (NDVI z-units) 0.617
RMSE reduction −3.2 %
posterior SD / prior SD 0.89

A pilot run without localization (localization_radius = NULL) exhibited
the textbook small-ensemble collapse pathology — analysis RMSE growing
above prior RMSE even as the posterior spread shrank to 5 % of prior —
and is kept in the vignette as the motivation for the taper.

Deliverables

  • R/temporal_kalman.R — the stochastic EnKF implementation.
  • data-raw/temporal_cerrado_prepare.R — one-time 4D cube builder
    (MODIS per-cell .rds cache for resume-safe restart; POWER
    JSON cache for both PRECTOTCORR and T2M).
  • data-raw/temporal_cerrado_run.R — trains the K = 10 ConvLSTM
    ensemble, rolls the forecast forward, applies the localized
    Kalman update, slim-saves the bundle.
  • inst/extdata/temporal_cerrado_results.rds — 195 KB reproducible
    result bundle so the vignette builds offline.
  • Vignette pilar3-4d-real — end-to-end narrative with rollout
    RMSE, posterior ensemble-mean / uncertainty maps, gain diagnostics
    and a discussion of the ensemble-collapse pathology.

Quality

  • 19 new tests (test-temporal-kalman.R) covering tight / loose
    obs, 3-D and 4-D input, multi-obs shape, 4-D time_step slicing,
    degenerate single-member case, input validation, and the
    Gaspari–Cohn zero-outside-2R property.
  • R CMD check: 0 errors / 0 warnings / 1 unrelated NOTE
    (future file timestamps — system-clock artefact).

edaphos 1.4.0 — Pillar 1 on real Cerrado data

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 01:21

Summary

Takes Pillar 1 from the synthetic br_cerrado toy to real
Brazilian Cerrado data
. Backdoor-adjusted direct effects on
1 095 WoSIS topsoil profiles, with a DAG built against the exact
v1.3.1 covariate stack and block-bootstrap confidence intervals that
respect the spatial clustering of the profiles.

New API

  • causal_cerrado_real_dag() — a DAG over 12 nodes / 23
    directed edges on the column names of the v1.3.1 case-study
    bundle, encoding: relief → climate (orographic + lapse-rate) →
    land cover; relief → texture → density; climate + texture +
    slope + land cover → SOC.

Identification of direct effects uses the existing
causal_adjustment_set() + causal_estimate_effect() API — no new
estimator code, just a domain-specific DAG and a real dataset.

Identified direct effects (LM, block-bootstrap by cluster, B = 200)

Exposure Naive slope Identified direct Bootstrap 95 % CI
wc_bio_12 (MAP, g/kg per mm) +0.0072 +0.0071 [+0.0002, +0.0121]
wc_landcover_trees (g/kg per % trees) +0.898 +2.048 (2.3×) [−0.465, +6.901]
soilgrids_clay (g/kg per % clay) +0.526 +0.195 (0.37×) [−0.099, +0.688]

Confounding moves in both directions. Naive OLS under-estimates
the land-use causal effect by more than half (tree cover really
does matter in Cerrado SOC) and over-estimates clay's direct
effect by nearly 3× (its apparent SOC lift is mostly slope / texture
confounding). Without DAG-guided adjustment all three numbers would
be reported wrong — exactly the kind of mistake Pearl's framework
is supposed to prevent, now demonstrated on a real Brazilian
dataset instead of a synthetic cube.

Deliverables

  • data-raw/causal_cerrado_real.R — fully reproducible analysis.
  • inst/extdata/causal_cerrado_real.rds — 62 KB slim results
    bundle.
  • Vignette pilar1-causal-real — narrative walk-through with the
    DAG rendered via ggdag and the naive-vs-identified summary
    table.

Quality

  • 11 new tests pinning the DAG structure and the adjustment sets
    for all three exposures.
  • R CMD check --as-cran: 0 errors / 0 warnings / 2 harmless
    NOTEs
    .

edaphos 1.3.1 \u2014 honest Cerrado benchmark repair

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 00:44

Summary

v1.3.0 shipped the first real-data benchmark but four load-bearing
defects made the numbers undersell the stack (R² = 0.24 for B1, E
worse than B1). v1.3.1 repairs each defect transparently, with the
repaired numbers defensible against the published literature.

What changed

  • Target: from "any horizon with lower_depth ≤ 30" (mixing
    0–5 / 5–15 / 15–30 cm slices) to "shallowest surface-anchored
    horizon (upper_depth == 0, lower_depth ∈ [5, 30] cm)" per
    profile. One physical quantity across every row.
  • Positional uncertainty: relaxed from ≤ 500 m to ≤ 2 km,
    matching the 1 km covariate resolution.
  • Covariates: added ESA WorldCover 2020 fractional covers
    (Zanaga et al. 2021) and 19 WorldClim 2.1 bioclim indices (Fick
    and Hijmans 2017). 32 → 56 covariates.
  • Evaluation: replaced the single 80/20 split with
    5-fold spatial cross-validation (k-means on coordinates).
    Every profile is a held-out prediction exactly once.
  • Tried and abandoned honestly: an integrated 0–30 cm SOC
    stock target. WoSIS's per-horizon bulk density covers only
    ~20 % of Brazilian profiles; the stock formulation degenerated
    into a constant-BD-fallback target with weaker signal than the
    plain concentration. Documented in the vignette and NEWS.

Headline numbers (5-fold CV, 1095 profiles)

Method n RMSE (g/kg) PICP @ 95 Interval score
B1 ranger QRF 1095 13.51 0.219 0.944 65.8
B2 ranger + gstat kriging 910 13.86 0.233 0.817 99.5
E ranger + MoCo v1 embed 923 14.07 0.157 0.940 71.7

R² 0.22-0.23 is in line with published Brazilian Cerrado DSM
(Gomes et al. 2019: 0.13 Brazil-wide; Nakhavali et al. 2018: 0.28
savanna with 200 profiles). The plain QRF is the calibration
champion
(PICP 0.944 at a 0.95 nominal level, best interval
score).

Still pending

The foundation-model embedding (encoder v1, 20 k InfoNCE steps)
trails B1. Encoder v2 with 200 k steps (10× budget) is
currently in training on Apple M1 Max MPS; v1.3.2 will re-run the
benchmark with the v2 weights and publish a new Zenodo deposit.

Scripts

  • data-raw/case_cerrado_prepare.R + _run.R — fully reproducible
    pipeline (~1 h end-to-end).
  • data-raw/pretrain_cerrado_train_v2.R — reproducible 200 k step
    encoder retrain.

Quality

  • Vignette case-cerrado-end-to-end rebuilt from scratch against
    the 1095-profile CV bundle.
  • R CMD check --as-cran: 0 errors / 0 warnings / 1 harmless
    NOTE
    .

edaphos 1.3.0 — honest benchmark on 1212 real Cerrado profiles

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 23:13

Summary

Closes the single biggest honesty gap in the README: the
"beyond regression-tree state-of-the-art" claim had never been
rigorously tested against a real dataset with a held-out test set.
This release provides that test on 1212 real Brazilian Cerrado
topsoil profiles
, and adjusts the README to reflect what the
numbers actually say.

The benchmark

Setting. On an 80 / 20 spatial train/test split (961 train, 240
test, stratified by 2×2 longitude × latitude quadrants so the test
set always contains points from every sub-region of the biome),
three competing stacks are evaluated with the same covariates,
seeds, split and metrics:

  • B1 ranger quantile regression forest on the raw 32-layer
    SoilGrids + WorldClim + SRTM covariate stack.
  • B2 B1 + gstat residual kriging — the canonical
    Hengl-style classical DSM recipe.
  • E B1 + the 64-dim MoCo v2 embedding from the publicly
    released edaphos-cerrado-moco-v1 encoder (Zenodo DOI
    10.5281/zenodo.19701276).

Results on the held-out test set:

method n RMSE (g/kg) MAE PICP @ 95 Interval score
B1 ranger — raw covariates 240 12.28 7.51 0.24 0.946 57.5
B2 ranger + gstat kriging 141* 11.51 7.18 0.09 0.738 104.1
E ranger + MoCo v2 embedding 240 12.53 8.28 0.21 0.858 85.5

* Kriging returned NA outside the fitted variogram's effective range;
the reduced n is itself a lesson.

Honest reading. Plain QRF is the calibration champion (PICP
0.946 at a 0.95 nominal level; best interval score). Residual
kriging lowers point RMSE but blows up calibration and drops data
far from the variogram range. The foundation-model embedding does
not beat B1 on this AoI
— the encoder was trained for only 20 k
InfoNCE steps on a smaller core-Cerrado AoI, and the raw covariate
stack is already rich enough that the embedding adds little marginal
signal over it. The Pillar 4 payoff is expected when the raw stack
is thinner (SAR-only or MODIS-only regions), which is the v1.4.0
agenda.

Data sources (all real, all open-licensed, all attributed)

  • Cerrado biome polygon — IBGE 1:250 000 Biomes via geobr
    (Pereira & Gonçalves 2019).
  • SOC observations — WoSIS snapshot 2019 (Batjes, Ribeiro and
    van Oostrum 2020, DOI
    10.5194/essd-12-299-2020,
    CC-BY-4.0), fetched live from
    ISRIC WFS.
  • Covariates — SoilGrids 250 m
    (Hengl et al. 2017),
    WorldClim 2.1
    (Fick & Hijmans 2017),
    SRTM 30-arcsec (Jarvis et al. 2008), all via geodata.

Deliverables

  • New vignette case-cerrado-end-to-end — narrative walk-through
    with per-dataset profile counts, a stratified-split map,
    observed-vs-predicted scatter (per method), PICP + interval-width
    bars, and residual geography. Every plot uses real data; every
    claim is sourced to a DOI / URL.
  • R/edaphos_metrics.R — standardised edaphos_rmse() /
    edaphos_mae() / edaphos_r2() / edaphos_bias() /
    edaphos_picp() / edaphos_interval_score() /
    edaphos_ece() / edaphos_metrics_summary() helpers. 21 new
    unit tests
    (all green on hand-computed fixtures).
  • data-raw/case_cerrado_prepare.R + _run.R — fully
    reproducible benchmark pipeline (~1 h end-to-end, ~2 GB
    downloads).
  • inst/extdata/case_cerrado_results.rds — the pre-computed
    benchmark results shipped with the package so the vignette
    builds on any installation without running the heavy prep.
  • README Benchmarks section at the top of the file, with the
    honest table and the "what we learned" paragraph.

New Suggests

geobr (IBGE biome polygon), dplyr and patchwork (vignette
plotting stack).

Quality

  • 21 new tests for edaphos_metrics (hand-computed fixtures).
  • Full test suite passes on all pillars.
  • R CMD check --as-cran: 0 errors / 0 warnings / 1 harmless
    NOTE
    (torch tempdir).

edaphos 1.2.0 — Pillar 4 foundation model reaches the user

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 22:01

Summary

Promotes Pillar 4 from a self-supervised-training scaffold to a
ready-to-use transfer-learning pipeline. Three capabilities land
together: a downstream fine-tuning API, a public pretrained encoder
hosted on Zenodo, and GPU (Apple Silicon MPS / NVIDIA CUDA) dispatch.

Downstream fine-tuning API

Two new entry points attach a classification or regression head on
top of any MoCo v2 / SimCLR encoder and train it against a labelled
patch set:

  • foundation_fit_classifier() — linear probe (frozen backbone)
    or full fine-tuning with a two-group learning-rate schedule
    (Kornblith, Shlens and Le 2019). predict(..., type = "class")
    or type = "prob".
  • foundation_fit_regressor() — same API for continuous
    targets; target normalisation is handled internally (center + scale
    at fit time, un-scale at predict) and a Huber loss option is
    provided for robustness against pedogenic outliers.

Both dispatch via device = c("cpu", "mps", "cuda") and fall back
to CPU with a clear message when the requested backend is
unavailable.

Public pretrained encoder: edaphos-cerrado-moco-v1

The first published Pillar 4 foundation model is live on Zenodo:

It is a MoCo v2 (He et al. 2020; Chen et al. 2020) encoder
pretrained for 20 000 InfoNCE steps on an Apple Silicon M1 Max (MPS
backend) over 50 000 16×16 raster patches sampled from a core
Cerrado AoI (longitude −53 to −43, latitude −23 to −10). Input
channels (31 total) are a multi-source stack aligned to a 0.01
degree (~1 km) grid:

Source Channels
SoilGrids 250m (0–5 cm mean) SOC, clay, sand, pH(H₂O), bulk density
WorldClim 2.1 (Brazil country pack) 12 monthly precipitation + 12 monthly mean temperature
SRTM 30 arc-second elevation, slope

Architecture: 5-block convolutional backbone → 64-dimensional
feature → 2-layer MLP projection head (proj_dim = 32). Training
used a queue of 4096 negatives, InfoNCE temperature 0.07, momentum
0.999, Adam learning rate 3e-4, batch size 64. Final InfoNCE loss
~1.64 (from an initial ~7.68 — a 4.7× reduction).

Weights distribution infrastructure

Three new functions consume the encoder registry:

  • foundation_weights_list() — catalogue of every published
    encoder (name, DOI, SHA-256, AoI, channel count, feature
    dimension, edaphos version).
  • foundation_weights_download() — fetches from Zenodo, verifies
    the SHA-256 digest, caches under
    tools::R_user_dir("edaphos", "cache") / weights /.
  • foundation_weights_load() — rebuilds the in-memory
    edaphos_foundation_moco wrapper with the state dict loaded and
    shape metadata populated.
moco <- foundation_weights_load("edaphos-cerrado-moco-v1")
fit  <- foundation_fit_classifier(moco, labelled_patches, soil_order,
                                    epochs = 40L, device = "mps")

Reproducibility

Two scripts under data-raw/ rebuild the deposit from scratch on
any R ≥ 4.3 + torch ≥ 0.16 machine:

  1. pretrain_cerrado_prepare.R — downloads the SoilGrids, WorldClim
    and SRTM tiles; crops them to the AoI; aligns them to the
    analysis grid; samples 50 000 patches.
  2. pretrain_cerrado_train.R — runs 20 000 MoCo v2 steps on the
    requested backend; writes the state dict + metadata + loss
    history; computes the SHA-256 for registry verification.

The seed is fixed (seed = 2026L) so re-running on the same
hardware produces byte-identical artefacts. The seed, the AoI, the
variable list, the alignment grid and the training hyperparameters
are all recorded in the Zenodo deposit's metadata.json.

A companion data-raw/zenodo_upload.R is the reproducible
create / publish / discard pipeline that produced this deposit and
will produce every subsequent one.

Bug fixes

  • foundation_moco_embed() now forces the encoder into eval()
    mode before the forward pass, so BatchNorm uses its saved
    running_mean / running_var rather than batch-level statistics.
    Previously the embeddings depended on the current batch
    composition and disagreed with any reloaded copy of the same
    encoder — the bug that would have made the Zenodo round-trip
    miscompare.
  • foundation_tile_source_soilgrids() now accepts both the
    human-readable depth strings ("0-5cm", "5-15cm", …) documented
    in its @param block and the integer form
    geodata::soil_world() expects internally.

Quality

  • 49 new tests (test-foundation-finetune,
    test-foundation-weights) land green.
  • Full Foundation test suite: 94 / 94 green.
  • R CMD check --as-cran: 0 errors / 0 warnings / 1 harmless NOTE.

edaphos 1.1.0 — Bayesian PIML, BatchBALD, structure learning, LLM voting

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 17:18

Summary

Three pillars gain scientifically load-bearing depth in one coherent
release: point estimates become posteriors in Pillar 2, heuristic
batch acquisition becomes information-theoretic in Pillar 5, and the
LLM-only causal-discovery story in Pillar 1 acquires both a
bottom-up counterpart and a multi-extractor consensus layer.

Pillar 2 — Bayesian posterior over the pedogenetic ODE

piml_profile_fit_bayesian() returns the full posterior over
$(\lambda_0, \mu, y_\infty, y_0)$ via two nested approximations:

  • Laplace (default, O(ms) per pedon) — Gaussian posterior from
    the MAP + inverse observed Fisher information at the MAP (Bishop
    2006, §4.4), with 2 000 pre-sampled draws for downstream
    predict().
  • Adaptive random-walk Metropolis (Haario, Saksman and Tamminen
    2001; ~seconds per pedon) — proposal covariance starts at the
    Laplace covariance, scaled by the Roberts–Gelman–Gilks
    $(2.38)^2 / d$ factor, and is updated online by Haario recursion
    after warm-up, so multimodal and non-Gaussian posteriors are
    captured faithfully.

predict.edaphos_piml_bayes() propagates the full posterior through
the forward ODE; include_obs_noise = TRUE switches the credible
interval from "mean function" to "future observation" semantics.

The Neural-ODE analogue is piml_neural_ode_fit_ensemble() — a
deep ensemble (Lakshminarayanan, Pritzel and Blundell 2017; Wilson
and Izmailov 2020) whose K independent networks' empirical spread
approximates the Bayesian predictive posterior.

Pillar 5 — BatchBALD information-theoretic batch acquisition

al_query_batchbald() implements BatchBALD (Kirsch, van Amersfoort
and Gal 2019) for regression on top of the existing QRF backbone.
The trees of the forest are the T posterior parameter draws, the
joint epistemic covariance is their empirical covariance across
candidates, and the mutual-information objective reduces to

$$ \mathrm{BatchBALD}(B) ;\propto; \tfrac{1}{2}\log\det!\bigl( \mathrm{Cov}_\theta(f_\theta(B)) + \sigma_a^2 I_{|B|} \bigr). $$

Greedy argmax via incremental Cholesky / Schur-complement updates
gives every greedy step $O(m^2 n_{\mathrm{pool}})$ rather than
$O(m^3 n_{\mathrm{pool}})$; the log-det is monotone submodular so
greedy selection inherits a $(1 - 1/e)$ optimality guarantee
(Nemhauser, Wolsey and Fisher 1978). BatchBALD addresses the
cluster-of-near-duplicates failure mode of top-$n$ BALD and
complements — does not replace — the uncertainty + diversity hybrid
for physics-gated, cost-aware acquisition.

Pillar 1 — Structure learning + multi-extractor voting

Structure learning. causal_structure_learn() wires four
bnlearn algorithms through a uniform interface that returns an
edaphos_causal_kg:

  • "hc" — hill-climbing over Gaussian BIC (default).
  • "tabu" — tabu-search hill-climbing.
  • "pc-stable" — PC-stable constraint-based (Colombo and Maathuis
    2014).
  • "mmhc" — max-min hill-climbing hybrid (Tsamardinos, Brown and
    Aliferis 2006).

Whitelists / blacklists encode pedological priors ("parent material
must precede soil chemistry"); an optional non-parametric bootstrap
attaches per-edge strengths that become the confidence field of
the returned KG, so the learned DAG can be unioned with the
LLM-derived KG via the standard causal_augment_dag() path.

Multi-extractor consensus. causal_llm_vote() runs N LLM
backends on the same abstract and resolves disagreements by one of
three voting rules:

rule keeps edges asserted by
"majority" at least min_support backends (default ceil(N/2))
"weighted" edges with $\sum_i w_i c_i \geq$ threshold
"intersection" every backend

causal_llm_ingest_abstract_voted() wraps vote + KG insertion and
tags the source field with the vote metadata. A crashing backend
emits a warning and contributes zero claims — the vote continues
with the remaining backends so corpus-scale ingestion degrades
gracefully.

Documentation

  • Vignette pilar1-causal gains §12 "Structure learning from
    horizon data" and §13 "Multi-extractor consensus: voting across
    LLM backends" with reproducible examples.
  • Vignette pilar2-piml-profile gains §8 "Bayesian posterior over
    the ODE parameters" with Laplace + MCMC + deep ensemble
    derivations.
  • Vignette pilar5-active-learning gains §6 "Information-theoretic
    batch acquisition: BatchBALD" with the log-det derivation and a
    cluster-of-near-duplicates motivating example.
  • README gains a per-addition subsection in each affected pillar.

Quality

  • 89 new tests across five files (test-piml-bayesian,
    test-piml-neural-ode-ensemble, test-al-batchbald,
    test-causal-structure, test-causal-llm-vote). The LLM-voting
    tests use testthat::local_mocked_bindings() so the multi-
    backend path is exercised against deterministic fixtures.
  • R CMD check --as-cran: 0 errors / 0 warnings / 0 notes.

Install notes

New optional dependency for structure learning:

install.packages("bnlearn")

bnlearn is a Suggests; every entry point emits a clear install
hint if its stack is missing.

edaphos 1.0.0 — Pillar 1 at paper scale: persistence, RDF export and audit

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 16:18

Summary

First major version: a Knowledge Graph built from tens of thousands
of abstracts is no longer an in-memory dead end. Four additive
capabilities turn Pillar 1 into a research artefact that can be
persisted, federated with external RDF vocabularies, and
interrogated by standard audit primitives.

Persistence

  • causal_kg_save() / causal_kg_load() serialise an
    edaphos_causal_kg through its tidy edge list (not through
    igraph's raw C-level pointer layout), so the resulting .rds
    is portable across igraph versions and byte-reproducible.
    Loading reruns the duplicate-edge merge + cycle check so a saved
    KG round-trips exactly to a freshly built one. The header
    carries a format_version tag, a saved_at timestamp and the
    edaphos / R version metadata.

RDF 1.1 Turtle export

  • causal_kg_to_turtle() emits a W3C-conformant RDF 1.1 Turtle
    document (Beckett et al. 2014). Each edge becomes a reified
    rdf:Statement carrying confidence, evidence, source(s) and
    timestamp, so the full provenance stack survives the round-trip;
    each node is assigned a stable IRI inside a user-controlled
    namespace so the KG federates cleanly with AGROVOC, ENVO and
    other SKOS thesauri via owl:sameAs. The emitter is pure R — no
    RDF library dependency — and the output parses against any
    RDF 1.1-conformant consumer (rdflib, Jena, Oxigraph, Blazegraph,
    GraphDB, Virtuoso).

Paper-scale audit

  • causal_kg_rank_edges(by = c("n_sources", "mean_confidence", "agrovoc_support")) collapses the KG to unique
    (cause, effect) pairs and sorts by a priority list of metrics.
    The single most informative signal for an LLM-extracted KG is
    n_sources — an edge asserted by 50 papers is far more
    trustworthy than one asserted by 1. An optional alignment
    argument attaches agrovoc_cause, agrovoc_effect and
    agrovoc_support columns so the caller can prefer edges whose
    endpoints resolve to community-governed vocabulary.

  • summary.edaphos_causal_kg() is the one-line health check:
    node count, edge count, unique source count, confidence
    quartiles, DAG-ness verdict, and the most prolific source.

Concurrent AGROVOC alignment

  • causal_ontology_agrovoc_align_batch() dispatches N
    single-term SPARQL queries concurrently through
    httr2::req_perform_parallel() with a user-controlled
    max_active, idempotent on-disk cache, and exponential-backoff
    retries. Measured speedup of roughly 5× at max_active = 5
    against agrovoc.fao.org, with warm-cache re-runs ~3700× faster
    than cold.

    A note on the batching strategy: true SPARQL-level batching
    (VALUES + CONTAINS(?label, ?term)) is rejected by AGROVOC's
    production endpoint with a 504 gateway timeout because the
    substring filter cannot short-circuit against a bound term set.
    agrovoc_align_batch() therefore batches at the transport layer
    instead — identical on-wire semantics, only the wall-clock time
    changes.

  • causal_kg_alignment(kg, vocab = "agrovoc", agrovoc_batch = TRUE, agrovoc_max_active = 5L) is the KG-level one-liner that
    routes the batched resolver through the standard alignment
    output.

Documentation

  • Vignette pilar1-causal gains §11 "Paper-scale audit:
    persistence, Turtle, ranked edges" deriving each primitive from
    first principles with reproducible toy-KG examples.
  • README gains a "Paper-scale persistence, Turtle export and
    multi-source audit" subsection with a complete save → Turtle →
    align → rank walk-through.

Quality

  • 62 new tests land in test-causal-v10.R covering the RDS
    round-trip, the Turtle emitter (prefix declarations, reified
    edges, escape rules, empty-KG handling), the ranker
    (multi-key sort, AGROVOC support, top_n), summary() and the
    mocked parallel-dispatch variant.
  • Full causal test suite: 167 / 167 green.
  • R CMD check --as-cran: 0 errors / 0 warnings / 2 harmless
    NOTEs (network timestamp + PySCF tempdir carried over from
    v0.9.0).

edaphos 0.9.0 — Pillar 6 meets real hardware and real chemistry

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 15:28

Summary

Promotes Pillar 6 (Quantum ML) from a statevector-only demonstration
to a NISQ-honest, chemistry-grounded pipeline. Three strictly
additive capabilities land together because each one only becomes
meaningful in the presence of the other two.

Shot-based execution

quantum_vqe_fit(backend = "aer_shots", shots = ...) wires
qiskit_aer.primitives.EstimatorV2 through the VQE loop. The
EfficientSU2 ansatz is transpiled to the standard
{id, rz, sx, x, cx, u} basis gate set so Aer can dispatch it, and
each energy evaluation is reconstructed from shots Monte-Carlo
samples with the expected 1/sqrt(shots) noise floor. SPSA
(Spall 1998) becomes the recommended optimiser under a stochastic
cost function.

Full IBM Quantum Runtime dispatch

backend = "ibmq" routes the VQE through
qiskit_ibm_runtime.EstimatorV2 inside a managed Session. The
ansatz is ISA-transpiled against the target backend's coupling map
via the preset pass manager (level 1), and the observable is
re-indexed with SparsePauliOp.apply_layout(). The new
mitigation argument maps {"none", "m3", "zne"} to IBM
resilience_level ∈ {0, 1, 2} (Kim et al. 2023, Nature 618,
500–505):

mitigation Technique
"m3" TREX + Matrix-free Measurement Mitigation for readout errors
"zne" Zero-Noise Extrapolation over gate-folding scales

Plumbing additions:

  • quantum_ibmq_submit() — low-level single-PUB primitive for
    custom hybrid loops that do not want quantum_vqe_fit()'s
    COBYLA/SPSA wrapper.
  • quantum_ibmq_least_busy() — automatic backend selection.
  • quantum_ibmq_backends(operational_only, simulator) — richer
    filtering.

qiskit-nature bridge

quantum_hamiltonian_from_pyscf() turns a molecular XYZ geometry
into a qubit Hamiltonian through the four-stage pipeline

XYZ → PySCF RHF → FreezeCore → ActiveSpace(n_e, n_o) → ParityMapper

and returns an edaphos_quantum_hamiltonian_nature that carries
the nuclear-repulsion, frozen-core and active-space energy shifts
as attributes so that quantum_nature_total_energy() can
reconstruct the full molecular energy from a VQE active-space fit.

Three curated organo-mineral presets ship via
quantum_hamiltonian_organo_mineral_nature():

Variant Pedological role Active space Qubits
"formic_acid" carboxylate –COOH — dominant humic functional group (2e, 2o) 2
"methanediol" ortho-diol — catechol-style Fe(III) chelator (2e, 2o) 2
"ferric_formate" monodentate Fe(III)–OOCH — minimum viable organo-mineral (4e, 4o) 4

On formic acid the (2e, 2o) active-space VQE recovers ~34
milli-Hartree of correlation energy below the Hartree–Fock
reference — the canonical signature that the quantum circuit is
genuinely beating the mean-field baseline.

Documentation

  • Vignette pilar6-quantum gains three new sections deriving the
    shot-based execution (§8), IBMQ + M3/ZNE mitigation (§9) and
    qiskit-nature organo-mineral Hamiltonians (§10) from first
    principles, and updates the roadmap to reflect v0.9.0 delivering
    the three items that were open at v0.8.0.
  • README Pillar 6 section gains a VQE-to-hardware walk-through and
    a first-principles organo-mineral subsection.

Quality

  • 20 new tests land in test-quantum-v09.R covering the shot-based
    VQE on H₂, the qiskit-nature formic-acid pipeline (both the
    Hamiltonian construction and total-energy reconstruction), the
    IBMQ preflight logic, and the mitigation ↔ resilience_level
    mapping.
  • Full quantum test suite: 89/89 green.
  • R CMD check --as-cran: 0 errors / 0 warnings / 2 harmless NOTEs
    (network timestamp + PySCF's own tempdir).

Install notes

New optional Python dependencies for the new features:

reticulate::py_install(
  c("qiskit-nature", "pyscf", "qiskit-ibm-runtime"),
  pip = TRUE
)
Sys.setenv(IBMQ_TOKEN = "<your-ibm-quantum-api-token>")  # only for backend = "ibmq"

All three packages are optional at load time; each public entry
point emits a targeted install-hint if its stack is missing.

edaphos 0.8.0 — Pillar 1 at literature scale

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 22 Apr 13:30

Summary

Promotes Pillar 1 (Causal AI) from a ten-abstract demo to a
literature-scale extractor. The LLM + Knowledge-Graph pipeline now
handles tens of thousands of abstracts end-to-end.

New features

  • Paginated corpus clients. causal_corpus_openalex() cursor-pages
    through the OpenAlex Works API transparently past the 200-per-page
    cap. causal_corpus_deduplicate() collapses DOI + title overlap
    between SciELO and OpenAlex.

  • Resumable cached ingestion. causal_llm_ingest_corpus() gains
    cache_dir + max_retries. Per-abstract tools::md5sum() keys each
    JSON cache entry; cache hits short-circuit the LLM call so
    interrupted runs resume exactly where they stopped. Malformed
    responses trigger exponential backoff; exhausted rows land in
    attr(kg, "failed"). A txtProgressBar tracks runs of ≥ 5 rows.

  • Live AGROVOC SPARQL alignment. causal_ontology_agrovoc_align()
    and causal_kg_alignment(vocab = "agrovoc") resolve KG labels
    against FAO's live AGROVOC endpoint, picking the
    Levenshtein-nearest skos:prefLabel and caching resolutions to disk.

Bundled live corpus

  • inst/extdata/cerrado_claims_real_corpus.jsonl — 210 causal claims
    extracted by Ollama + Gemma-4 from 100 deduplicated OpenAlex
    abstracts across three Cerrado queries, aligned against AGROVOC.
    Reproduced end-to-end by data-raw/run_large_corpus.R.

Documentation

  • Vignette pilar1-causal gains §10 "Scaling to tens of thousands of
    abstracts" deriving the pagination, cache, and AGROVOC alignment
    primitives from first principles.
  • README + roadmap + package doc updated to reflect the upgrade.

Quality

  • 105 / 105 causal tests pass (17 new scale tests covering pagination,
    cache hits, retry on malformed responses, AGROVOC Levenshtein
    selection, and kg_alignment(vocab = "agrovoc") plumbing).
  • R CMD check --as-cran clean: 0 errors / 0 warnings / 1 harmless
    "unable to verify current time" NOTE.

🤖 Generated with Claude Code