edaphos 1.3.1 \u2014 honest Cerrado benchmark repair
Summary
v1.3.0 shipped the first real-data benchmark but four load-bearing
defects made the numbers undersell the stack (R² = 0.24 for B1, E
worse than B1). v1.3.1 repairs each defect transparently, with the
repaired numbers defensible against the published literature.
What changed
- Target: from "any horizon with
lower_depth ≤ 30" (mixing
0–5 / 5–15 / 15–30 cm slices) to "shallowest surface-anchored
horizon (upper_depth == 0,lower_depth ∈ [5, 30] cm)" per
profile. One physical quantity across every row. - Positional uncertainty: relaxed from ≤ 500 m to ≤ 2 km,
matching the 1 km covariate resolution. - Covariates: added ESA WorldCover 2020 fractional covers
(Zanaga et al. 2021) and 19 WorldClim 2.1 bioclim indices (Fick
and Hijmans 2017). 32 → 56 covariates. - Evaluation: replaced the single 80/20 split with
5-fold spatial cross-validation (k-means on coordinates).
Every profile is a held-out prediction exactly once. - Tried and abandoned honestly: an integrated 0–30 cm SOC
stock target. WoSIS's per-horizon bulk density covers only
~20 % of Brazilian profiles; the stock formulation degenerated
into a constant-BD-fallback target with weaker signal than the
plain concentration. Documented in the vignette and NEWS.
Headline numbers (5-fold CV, 1095 profiles)
| Method | n | RMSE (g/kg) | R² | PICP @ 95 | Interval score |
|---|---|---|---|---|---|
B1 ranger QRF |
1095 | 13.51 | 0.219 | 0.944 | 65.8 |
B2 ranger + gstat kriging |
910 | 13.86 | 0.233 | 0.817 | 99.5 |
E ranger + MoCo v1 embed |
923 | 14.07 | 0.157 | 0.940 | 71.7 |
R² 0.22-0.23 is in line with published Brazilian Cerrado DSM
(Gomes et al. 2019: 0.13 Brazil-wide; Nakhavali et al. 2018: 0.28
savanna with 200 profiles). The plain QRF is the calibration
champion (PICP 0.944 at a 0.95 nominal level, best interval
score).
Still pending
The foundation-model embedding (encoder v1, 20 k InfoNCE steps)
trails B1. Encoder v2 with 200 k steps (10× budget) is
currently in training on Apple M1 Max MPS; v1.3.2 will re-run the
benchmark with the v2 weights and publish a new Zenodo deposit.
Scripts
data-raw/case_cerrado_prepare.R+_run.R— fully reproducible
pipeline (~1 h end-to-end).data-raw/pretrain_cerrado_train_v2.R— reproducible 200 k step
encoder retrain.
Quality
- Vignette
case-cerrado-end-to-endrebuilt from scratch against
the 1095-profile CV bundle. R CMD check --as-cran: 0 errors / 0 warnings / 1 harmless
NOTE.