Skip to content

MNiShed v3.0.0

Choose a tag to compare

@awickert awickert released this 23 Jun 20:36
· 23 commits to master since this release

TL;DR

Major release. The library was renamed from hydroRaVENS to MNiShed.
Several parameter, attribute, and config names changed (see the migration
table below). The numerical core moved to a Numba JIT time loop —
roughly 100×–400× faster than the pure-Python loop depending on record
length — installable via the new jit extra (pip install mnished[jit]). New mechanics: power-law (nonlinear) recession,
leakance + threshold junctions between reservoirs, constant-fraction
tile drainage and threshold-activated multipath parallel drainage,
and exponential PDM saturation-excess overland flow, plus snow
insulation, DTR-based frozen-ground decay, and flexible ET modules. A
CSDMS BMI wrapper (BmiMNiShed) was introduced with CSDMS Standard
Names–aligned variables. GitHub Actions CI added. Substantial
calibration-robustness and numerical-correctness fixes (water-balance
closure, deficit handling, NaN propagation, multi-decade chaining,
JIT↔pure-Python equivalence, and the mean-residence-time gauge).

Breaking changes (you'll need to update existing code/configs)

Old New Where it bites
Package name hydroRaVENS mnished imports, install: pip install mnished, from mnished import ...
BMI class BmiHydroRaVENS BmiMNiShed BMI users update the class name and import (from mnished import BmiMNiShed)
Reservoir(t_efold=...) / t_recession=... Reservoir(recession_coeff=...) constructor arg name
YAML e_folding_residence_times__days: recession_coefficients: all config files
Reservoir.H_infiltrated Reservoir.H_to_next attribute access
Boolean scale_et=True/False string enforce_water_balance='water-year' / 'global' / 'none' run_and_score() argument; YAML enforce_water_balance: key
evapotranspiration_method: ThorntwaiteChang2019 (misspelled) ThornthwaiteChang2019 YAML evapotranspiration_method: value
BMI variable names (atmosphere__temperature, daily min/max temperature, channel_exit_water__volume_flow_rate, ET forcing input) CSDMS-aligned (atmosphere_bottom_air__temperature, …__time_min_of_temperature / …__time_max_of_temperature, channel_exit_water_x-section__volume_flow_rate, …__uncorrected_evapotranspiration_volume_flux) BMI couplers update variable names

Migration: for an existing config + driver pair, do a global
search-and-replace on the old terms (t_efold, t_recession,
e_folding_residence_times__days, H_infiltrated, and the misspelled
ThorntwaiteChang2019ThornthwaiteChang2019), and re-spell
scale_et=True/False as the corresponding enforce_water_balance
string. BMI couplers update the variable names above. Note also that
Buckets.initialize() now raises FileNotFoundError / yaml.YAMLError
on a missing or unparseable config file instead of calling sys.exit(), so
library, BMI, and notebook callers can handle the error (the mnished CLI
still exits cleanly).

New features

Numerical core

  • Numba JIT-compiled time loop (_jit_run). The daily loop is roughly
    100×–400× faster than the pure-Python implementation, scaling with
    record length (~120× on a 3-year run, ~300–400× on multi-decade to
    century-long daily records — see the Performance page in the documentation
    for the full scaling analysis and benchmark scripts). Used automatically
    when Numba is installed; install it with the jit extra (pip install mnished[jit]). The pure-Python loop is retained as a fallback for
    non-Numba environments and for code paths that don't yet fit into the JIT
    (PDM, ET water-stress).

Reservoir mechanics

  • Power-law (nonlinear) recession. recession_exponents per reservoir
    (default 1.0 = linear). Exact integration in the JIT loop;
    mean_residence_time() for physically-comparable timescales at a
    reference discharge.
  • Junction types between reservoirs. New junction_type argument
    on Reservoir and matching junction_types:, leakance_R__days:,
    H_threshold__mm: YAML keys:
    • fraction (default, current behaviour)
    • leakance — head-difference driven flow with leakance resistance R
    • threshold — dead-storage threshold; recession only above H_threshold
  • Tile drain sub-reservoir. f_tile + tau_tile divert a constant
    fraction of subsurface drainage to a fast linear sub-reservoir that
    drains directly to stream. Represents constant-fraction agricultural
    tile bypass.
  • Multipath threshold-activated parallel drain. New
    multipath_threshold + multipath_timescale on Reservoir; per-reservoir
    YAML lists multipath_thresholds__mm / multipath_timescales__days.
    Adds a second linear outflow path direct to stream when storage
    exceeds the threshold. Distinct from f_tile/tau_tile: it is
    state-dependent (activates only above the threshold), not a constant-
    fraction split. Models a tile-drain network at a fixed installation
    depth. Both tile mechanisms can be configured independently.
  • Exponential PDM saturation-excess overland flow. pdm_H0 opens up
    the probability-distributed-model fraction of recharge that is shed
    immediately to discharge from a fraction of catchment with
    exponentially-distributed storage capacities.

Hydrologic processes

  • Snowpack insulation in FGI. snow_insulation_k reduces effective
    air temperature reaching the soil under deep snowpack, with
    documented equifinality caveats.
  • DTR-based FGI decay. When daily Tmax/Tmin are available, FGI decay
    coefficient A_t scales with the fraction of the day above 0°C
    (Molnau & Bissell 1983); falls back to the constant A=0.97 when not.
  • Constant regional baseflow baseflow_Q for cases with sub-basin
    groundwater import.
  • Direct-runoff (Hortonian) bypass. direct_runoff_fraction (γ):
    fraction of positive recharge that bypasses the cascade. Reframed
    from a snow-coupled to a free fast-bypass parameter.
  • ET module flexibility. Three independent module toggles:
    • et_reservoir_draw: post-cascade ET drawn from reservoirs with
      optional wilting-point threshold (recommended for calibrations
      with a soil reservoir)
    • et_water_stress: storage-dependent ET scaling (et_scale,
      wp_soil, wp_soil_sigma)
    • et_scale is also usable as a free universal ET multiplier when
      neither stress module is enabled
  • Water-balance enforcement vocabulary. enforce_water_balance
    accepts 'water-year', 'global', 'none' instead of a single
    boolean. 'global' rescales ET by a single multiplier across the
    entire record (no hidden per-year DoFs that would inflate AIC).
  • Module enable/disable toggles. Top-level modules: block in
    YAML to switch snowpack, frozen_ground, rain_on_snow, direct_runoff,
    dtr_fgi_decay, et_water_stress, et_reservoir_draw on/off.
  • Auto-compute spin-up cycles. When spin_up_cycles=None,
    computed automatically as ceil(tau_max / record_length).
  • Analytical steady-state initial conditions. Used by default
    before spin-up to remove cycle-zero artifacts.
  • post_spinup_states hook. Inject per-decade initial conditions
    after spin-up; supports decade-chaining workflows where each new
    decade starts from the prior decade's end state.

Analysis and diagnostics

  • BrutsaertNieber recession analysis class. Fits −dQ/dt = a·Q^b to
    observed hydrograph recessions for B-N-style architecture diagnosis.
    Power-law goodness-of-fit diagnostics included.
  • HydrographSeparation class. Data-driven model initialization;
    time-domain recession analysis is now the primary timescale estimator,
    with the spectral (PSD) approach retained as a fallback/diagnostic;
    rolling-minimum τ_soil estimation, decoupled from LP separation.
  • Priors and suggest_priors. Suggested prior distributions for
    calibration parameters from observed-data diagnostics.
  • Composite scoring metrics. KGE_logKGE, KGE_logKGE_logFDC,
    KGE_logKGE_logFDC_BFI, logKGE_logFDC_BFI. The mean-of-multiple-
    KGEs framing avoids over-emphasizing high flows.
  • kge_logfdc diagnostic on CalibResult.
  • Reservoir.mean_residence_time() for nonlinear reservoirs:
    physically-comparable timescale at a reference steady-state discharge.

CSDMS BMI wrapper

  • BmiMNiShed class implementing the Community Surface
    Dynamics Modeling System Basic Model Interface.
  • 10-reservoir cap (up from 3) via reservoir_0 through reservoir_9.
  • channel_exit_water_x-section__volume_flow_rate BMI output in m³ s⁻¹.
  • Variable names reviewed against the current CSDMS Standard Names
    conventions: air temperature uses the atmosphere_bottom_air object
    (atmosphere_bottom_air__temperature, and
    …__time_min_of_temperature / …__time_max_of_temperature for the daily
    extremes). The ET forcing input is
    land_surface_water__uncorrected_evapotranspiration_volume_flux — a
    deliberately model-specific name for the ET as supplied (Thornthwaite or
    user series), before the water-balance correction — distinct from the
    corrected ET output land_surface_water__evapotranspiration_volume_flux.
  • Flux-partition, state, and ET outputs: direct-runoff, baseflow,
    tile-drain, and multipath *_volume_flux components, plus
    land_surface__frozen_ground_index and an actual-ET
    land_surface_water__evapotranspiration_volume_flux output. The
    baseflow term (baseflow_Q) is exposed as its own output and is not
    folded into the runoff total, so a coupler adds it explicitly. Note this
    differs from run_and_score, whose scored discharge does include
    baseflow_Q and Nash routing (routing_K) as output-layer
    post-processing; the streaming BMI applies neither (routing is an
    inherently batch operation), so a routed/baseflow-augmented calibration
    is not reproduced through the BMI unless the coupler reapplies them.
  • BMI tests skip gracefully if bmipy not installed; bmipy available
    as an optional bmi extra.

Calibration API

  • AIC k-counting for newly calibratable parameters:
    leakance_R_calibrated, H_threshold_calibrated,
    recession_exponents_calibrated. Counts free parameters
    independently of the lengths of the parameter lists.
  • Tile-share counting: the AIC k count for f_tile uses unique
    non-zero values (handles shared tile parameters across reservoirs).
  • Multipath calibration support (multipath_threshold,
    multipath_timescale, multipath_calibrated) in run_and_score().
  • Single full-record ET multiplier in decade mode. compute_ET()
    uses the full-record multiplier, not a decade-window-specific one,
    so individual decade calibrations don't drift relative to the global
    ET correction.

Bug fixes

  • Numba JIT correctness. The JIT loop dropped negative-ET
    (condensation) input to the soil reservoir, and omitted tile storage
    from the subsurface total on skipped (missing-forcing) days. Both now
    match the pure-Python loop, which is verified by JIT↔pure-Python
    equivalence tests.
  • recession_H_ref standardized to 1.0 and
    Reservoir.mean_residence_time() gauge-corrected — for nonlinear
    reservoirs it was off by the H_ref^((b−1)/b) factor; run_and_score
    no longer sets a hidden H_ref anchor.
  • ET-reservoir-draw condensation (negative ET) is now capped at
    Hmax, with the surplus shed to runoff instead of stored above the cap.
  • AIC free-parameter count counts only finite Hmax entries
    (inf = no cap, not a calibrated parameter).
  • H_deficit_carry is now correctly reset in three places where it
    could leak across spin-up/IC injection: before full-record spin-up,
    before pre-decade spin-up, and when initial_states/post_spinup_states
    are applied. Affected multi-decade chained calibrations.
  • f_to_discharge index alignment for sparse reservoir lists.
  • NaN propagation explicitly handled in three pathways: NaN forcing
    through compute_global_ET_multiplier / update(); NaN propagation
    through Nash cascade routing; NaN temperature explicitly carried
    through Thornthwaite ET.
  • compute_water_year() off-by-one for January start months.
  • compute_ET() idempotency: drops 'ET multiplier' before merging
    to avoid duplicated columns on repeat invocations.
  • ThornthwaiteChang2019 ET is robust to missing temperature normals
    and missing discharge.
  • land_surface__frozen_ground_index BMI output returns NaN before
    the first update(), consistent with the other outputs.
  • Removed stale code: Snowpack.discharge() and dead H_discharge
    attribute; redundant _timestep_i reset in run_and_score() spin-up
    loop; obsolete scalar_dt (daily timestep is now documented as a
    design choice).
  • Configuration robustness: fdd_threshold and recession_exponents
    YAML-configurable; YAML key fixes (__m__mm in several places).

Documentation

  • Comprehensive RTD updates to model_description, configuration,
    calibration, quickstart, tutorial, references, and BMI sections,
    reviewed end-to-end for v3.0.0 accuracy.
  • New sections: mean residence time computation, B-N recession
    analysis, power-law recession, PDM saturation-excess, reservoir
    junction types (fraction / leakance / threshold), tile drain
    vs multipath contrast, calibration prior estimation tutorial,
    post_spinup_states decade initial-storage calibration, an
    evapotranspiration-pathway diagram, a Numba/jit-extra note, and a
    Performance page with the Numba JIT scaling benchmark.
  • Performance benchmarks (benchmarks/bench_jit.py, plot_jit.py):
    reproducible scripts that time the JIT vs. pure-Python time loop across
    record lengths and produce the Performance-page figure (saving a
    commit-stamped results file under benchmarks/results/).
  • References cleaned up: all cited sources now in references.rst;
    M&B URL fixed; Bergström (1976) added; Molnau & Bissell (1983) credit
    added for exponential snow insulation.
  • README trimmed to a landing page; full docs at RTD.

Testing and CI

  • GitHub Actions CI runs the test suite, a Ruff lint job, and a
    build check on every push.
  • Numba JIT job (tests-jit) runs the suite with Numba on a pinned
    NumPy so the JIT path — including JIT↔pure-Python equivalence — is
    exercised in CI.
  • Core unit tests for Reservoir, Snowpack, BrutsaertNieber, Buckets,
    HydrographSeparation, suggest_priors / Priors, junction types, and the
    multipath drain; nonlinear-reservoir, T=0 snowpack-boundary, and
    dtype-safe water-balance tests.
  • NumPy 2.x in the CI test matrix.

Examples and tutorials

  • examples/ organised into cannon_forward/ (forward simulation)
    and cannon_inverse/ (Dakota-driven calibration), both verified to run
    against the v3.0.0 API.
  • Tutorial updated for the renamed package, recession_coeff,
    steady-state initialization, and MRT computation.

Acknowledgements

This release reflects collaborative work between A. Wickert and Claude
(Anthropic) on the v3.x architecture, including the multipath drainage
mechanism.


Version bump rationale

v3.0.0 (major version increment) because:

  • Package renamed (hydroRaVENS → MNiShed)
  • Breaking parameter, attribute, and config renames in user-facing API
    and YAML (including the BMI variable names)
  • Several behavioural defaults shifted (string enforce_water_balance
    vocabulary; auto-spin-up; analytical steady-state IC)

Backwards-compat shims are not provided — the rename is a clean break,
documented in the migration table above.