Skip to content

edaphos 1.3.1 \u2014 honest Cerrado benchmark repair

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 00:44
· 170 commits to main since this release

Summary

v1.3.0 shipped the first real-data benchmark but four load-bearing
defects made the numbers undersell the stack (R² = 0.24 for B1, E
worse than B1). v1.3.1 repairs each defect transparently, with the
repaired numbers defensible against the published literature.

What changed

  • Target: from "any horizon with lower_depth ≤ 30" (mixing
    0–5 / 5–15 / 15–30 cm slices) to "shallowest surface-anchored
    horizon (upper_depth == 0, lower_depth ∈ [5, 30] cm)" per
    profile. One physical quantity across every row.
  • Positional uncertainty: relaxed from ≤ 500 m to ≤ 2 km,
    matching the 1 km covariate resolution.
  • Covariates: added ESA WorldCover 2020 fractional covers
    (Zanaga et al. 2021) and 19 WorldClim 2.1 bioclim indices (Fick
    and Hijmans 2017). 32 → 56 covariates.
  • Evaluation: replaced the single 80/20 split with
    5-fold spatial cross-validation (k-means on coordinates).
    Every profile is a held-out prediction exactly once.
  • Tried and abandoned honestly: an integrated 0–30 cm SOC
    stock target. WoSIS's per-horizon bulk density covers only
    ~20 % of Brazilian profiles; the stock formulation degenerated
    into a constant-BD-fallback target with weaker signal than the
    plain concentration. Documented in the vignette and NEWS.

Headline numbers (5-fold CV, 1095 profiles)

Method n RMSE (g/kg) PICP @ 95 Interval score
B1 ranger QRF 1095 13.51 0.219 0.944 65.8
B2 ranger + gstat kriging 910 13.86 0.233 0.817 99.5
E ranger + MoCo v1 embed 923 14.07 0.157 0.940 71.7

R² 0.22-0.23 is in line with published Brazilian Cerrado DSM
(Gomes et al. 2019: 0.13 Brazil-wide; Nakhavali et al. 2018: 0.28
savanna with 200 profiles). The plain QRF is the calibration
champion
(PICP 0.944 at a 0.95 nominal level, best interval
score).

Still pending

The foundation-model embedding (encoder v1, 20 k InfoNCE steps)
trails B1. Encoder v2 with 200 k steps (10× budget) is
currently in training on Apple M1 Max MPS; v1.3.2 will re-run the
benchmark with the v2 weights and publish a new Zenodo deposit.

Scripts

  • data-raw/case_cerrado_prepare.R + _run.R — fully reproducible
    pipeline (~1 h end-to-end).
  • data-raw/pretrain_cerrado_train_v2.R — reproducible 200 k step
    encoder retrain.

Quality

  • Vignette case-cerrado-end-to-end rebuilt from scratch against
    the 1095-profile CV bundle.
  • R CMD check --as-cran: 0 errors / 0 warnings / 1 harmless
    NOTE
    .