Skip to content

edaphos 1.5.0 — Pillar 3 on real data + localized stochastic EnKF

Choose a tag to compare

@HugoMachadoRodrigues HugoMachadoRodrigues released this 23 Apr 13:52
· 168 commits to main since this release

Summary

Takes Pillar 3 from the synthetic temporal_synth_soc_cube() toy to
real Cerrado spatio-temporal data: a 10 × 10 × 168-month × 3-channel
cube over a 2° × 2° AoI (Goiás / Minas Gerais triple junction, Jan 2010 –
Dec 2023), and ships a stochastic Ensemble Kalman Filter with
optional Gaspari–Cohn localization for sequential assimilation of
new in-situ observations into a trained ConvLSTM forecast.

New API

  • temporal_kalman_update() — stochastic EnKF (Evensen 1994;
    Burgers, van Leeuwen & Evensen 1998) on a 3-D (N_ens, H, W) or
    4-D (N_ens, H, W, T) forecast ensemble. Returns the posterior
    ensemble, posterior mean / SD maps, per-observation Kalman-gain
    norm, and the innovation vector.
  • Optional localization_radius — Gaspari & Cohn (1999)
    5th-order polynomial taper on the gain, the standard small-ensemble
    fix (Houtekamer & Mitchell 2001) for spurious long-range
    correlations.

The 4D Cerrado cube

  • NDVI — MOD13Q1 250 m 16-day composites, aggregated to monthly
    means (NASA LP DAAC).
  • Precipitation — NASA POWER monthly mean daily precipitation
    (mm/day, MERRA-2 bias-corrected) scaled by days-in-month.
    (Original target was CHIRPS but the CHG data portal was returning
    HTTP 403 for every global_monthly/tifs/ request at the v1.5.0
    freeze; POWER is fully open and comparably accurate over the
    Cerrado.)
  • Air temperature — NASA POWER T2M (MERRA-2 bias-corrected),
    year-specific rather than a static climatology — an upgrade over
    the originally-planned WorldClim 2.1 pack, enabled by POWER
    returning both channels in one REST call per cell.

Rollout + assimilation result

Metric Value
K ensemble members 10
past window (training) Jan 2010 – Dec 2020 (132 mo)
future window (forecast) Jan 2021 – Dec 2023 (36 mo)
target month Dec 2023
in-situ observations assimilated 8
obs noise SD (NDVI z-units) 0.15
localization radius (cells) 2
prior RMSE (NDVI z-units) 0.637
analysis RMSE (NDVI z-units) 0.617
RMSE reduction −3.2 %
posterior SD / prior SD 0.89

A pilot run without localization (localization_radius = NULL) exhibited
the textbook small-ensemble collapse pathology — analysis RMSE growing
above prior RMSE even as the posterior spread shrank to 5 % of prior —
and is kept in the vignette as the motivation for the taper.

Deliverables

  • R/temporal_kalman.R — the stochastic EnKF implementation.
  • data-raw/temporal_cerrado_prepare.R — one-time 4D cube builder
    (MODIS per-cell .rds cache for resume-safe restart; POWER
    JSON cache for both PRECTOTCORR and T2M).
  • data-raw/temporal_cerrado_run.R — trains the K = 10 ConvLSTM
    ensemble, rolls the forecast forward, applies the localized
    Kalman update, slim-saves the bundle.
  • inst/extdata/temporal_cerrado_results.rds — 195 KB reproducible
    result bundle so the vignette builds offline.
  • Vignette pilar3-4d-real — end-to-end narrative with rollout
    RMSE, posterior ensemble-mean / uncertainty maps, gain diagnostics
    and a discussion of the ensemble-collapse pathology.

Quality

  • 19 new tests (test-temporal-kalman.R) covering tight / loose
    obs, 3-D and 4-D input, multi-obs shape, 4-D time_step slicing,
    degenerate single-member case, input validation, and the
    Gaspari–Cohn zero-outside-2R property.
  • R CMD check: 0 errors / 0 warnings / 1 unrelated NOTE
    (future file timestamps — system-clock artefact).