Skip to content

Uncertainty Quantification

Salvador Navas edited this page Aug 4, 2026 · 1 revision

Uncertainty Quantification

Package: pyhydra.uq

pyhydra.uq decides how to propagate uncertainty through an expensive hydraulic model: reduce the ensemble to a design and emulate the rest, or spend the whole simulation budget on direct Monte Carlo. The choice is made from a measurement -- taken on a pilot charged against the same budget -- rather than by convention.

import numpy as np
from pyhydra.uq import propagate

# X: the synthetic ensemble to propagate; simulate(): the expensive model
out = propagate(X, simulate, n_max=400, quantile=0.95)

print(out.strategy)                      # 'reduce_and_emulate' or 'monte_carlo'
print(out.decision.emulability.r2)       # what the pilot measured
print(out.estimate, out.n_evaluations)   # never more than n_max

Design Reduction and Weighting

Module: pyhydra.uq.design

Main APIs:

  • maxdiss_order
  • pilot_design
  • voronoi_weights
  • standardize

maxdiss_order is a generic, greedy maximum-dissimilarity selection (Camus et al., 2011) over an arbitrary input matrix, and is nested: the first m entries of a size-n order are themselves a valid size-m design. That nesting is what lets pilot_design charge a pilot against the budget once and reuse it as the opening of the final design if the run goes on to emulate.

voronoi_weights assigns each ensemble member to its nearest design point so error can be reported over the population a MaxDiss design was drawn from, rather than over the design itself, which over-samples the tails by construction.

This is a different function from pyhydra.climate.hybrid_downscaling.maxdiss, which is specific to synthetic flood events (one column treated as circular, one seed per hydrograph shape type) and returns a different order on the same data. The two are not interchangeable.

Emulability

Module: pyhydra.uq.emulability

Main APIs:

  • emulability
  • Emulability
  • SELECTION_CAP

emulability measures the best cross-validated R² attained by any member of a declared emulator family set, on a given design. It is a property of the triple (problem, family set, design size), not of the physical problem alone. The same procedure serves as the pilot diagnostic and as the selection step of the surrogate that is actually built, so the threshold it is compared against describes the model that ends up being used.

Emulator Families

Module: pyhydra.uq.emulators

Main APIs:

  • emulator_families
  • DEFAULT_FAMILIES
  • RBF, RBF_SMOOTHING
  • EnsembleMean
  • PeakOnly1D

The default family set is small and cheap: RBF interpolants and distance-weighted k-NN. EnsembleMean is the null model (an emulator that cannot beat it has learned nothing); PeakOnly1D is a monotone 1-D baseline standing in for the conventional practice of treating the response as a function of peak flow alone.

Strategy Rule

Module: pyhydra.uq.strategy

Main APIs:

  • select_strategy
  • propagate
  • reduce_and_emulate
  • StrategyDecision
  • PropagationResult
  • EMULABILITY_THRESHOLD

select_strategy compares the pilot's emulability against EMULABILITY_THRESHOLD (default 0.5) and returns a StrategyDecision, including margin and is_marginal for decisions taken on a hair's breadth of skill. propagate runs the whole thing end to end against a simulate callable and returns a PropagationResult.

Both branches are held to n_max solver evaluations in total: because maxdiss_order is nested, a run that goes on to emulate keeps its pilot points as the opening of the final design, while a Monte Carlo fallback has spent them and draws only n_max - n_pilot more.

Sequential Stopping

Module: pyhydra.uq.stopping

Main APIs:

  • sequential_design
  • loo_error
  • StoppingResult

sequential_design grows a MaxDiss design a few points at a time and tracks leave-one-out error, classifying each increment as improving, converged or deteriorating; only a run of consecutive converged increments stops it, so a flattened curve is not confused with one that has merely turned upward. Pass population= to weight the internal signal by Voronoi cell counts -- without it, the rule optimizes leave-one-out error over the design, which is not the population-weighted error the analysis is ultimately judged on.

Error Metrics

Module: pyhydra.uq.metrics

Main APIs:

  • error_metrics
  • weighted_rmse
  • weighted_r2

Every function takes an optional weight vector -- normally the Voronoi counts from voronoi_weights -- so weighted and unweighted numbers can both be reported.

What the Rule Is, and Is Not

select_strategy/propagate is a decision procedure with a measured failure rate, not an oracle. Tested prospectively on 1,620 synthetic problems against an independent reference, it selected the better of the two fixed policies in about 80% of runs and reduced mean error in the target quantile from 2.40% and 2.61% (the two fixed policies) to 1.70%; an oracle would have reached 1.09%.

It fails on responses that vary faster than the design can resolve -- sharp ridges, high-frequency oscillation -- where a pilot reports respectable skill precisely because it never samples the structure it is missing. When StrategyDecision.is_marginal is true, treat the choice as undetermined and, budget allowing, run both branches.

EMULABILITY_THRESHOLD was swept and reported against error in a high design quantile, where the loss curve is flat between roughly 0.4 and 0.7; re-sweep it for a different loss function.

Clone this wiki locally