CSPS

A multi-level assessment framework for Aquatic Ecosystem Models (AEMs)

CSPS is a framework for the hierarchical assessment of aquatic ecosystem models built on a range of metrics and characteristic signatures relevant to aquatic ecosystem condition. The framework is comprised of four levels: 0) conceptual validation; 1) comparison of simulated state variables with observations (‘state validation’); 2) comparison of fluxes (process rates) with measured fluxes (‘process validation’); and 3) comparison of system-level emergent properties, patterns and relationships (‘system validation’). Of these, only levels 0 and 1 are routinely undertaken at present. To highlight a diverse range of contexts relevant to the aquatic ecosystem modelling community, we present several case studies of improved validation approaches using the level 0-3 assessment hierarchy. It is our goal that the community–driven adoption of these metrics will lead to more rigorously assessed models, ultimately accelerating advances in model structure and function, and improved confidence in model predictions

Markdown

Markdown is a lightweight and easy-to-use syntax for styling your writing. It includes conventions for

Syntax highlighted code block

# Header 1
## Header 2
### Header 3

- Bulleted
- List

1. Numbered
2. List

**Bold** and _Italic_ and `Code` text

[Link](url) and ![Image](src)

For more details see GitHub Flavored Markdown.

Jekyll Themes

Your Pages site will use the layout and styles from the Jekyll theme you have selected in your repository settings. The name of this theme is saved in the Jekyll _config.yml configuration file.

Support or Contact

Having trouble with Pages? Check out our documentation or contact support and we’ll help you sort it out.

	Property being assessed	Description	Validation level	Typical range of data observation frequency	Spatial scale	Assessment technique (see Table 1)�	Comments (e.g. processing requirements)	Example references*
Water balance, waves & circulation
	Water level	Time-series comparison	1a	minutes-monthly	point; multiple points	E, R, V(TS), V(XY)	Direct observation or calculation from logged pressure gauge sensors, or from remote sensing approaches (e.g., satellite altimetry, radar)	Missaghi and Hondzo (2010)
		Tidal propagation	2b	minutes-hourly	horizontal transect	V(TX)	Magnitude of attenuation or amplification of tidal range within an estuary or coastal embayment, plotted as a function of distance
	Surface waves	Significant wave height	1a	seconds-minutes	point	V(TS), E, R	For models simulating surface waves the comparison of wave properties can be undertaken	Ji (2017)
		Wave length and period	1b	seconds-minutes	point	V(TS)
		Frequency spectra	1c	seconds-minutes	point	FFT, WT
	Evaporation	Time-series comparison	2a	minutes-daily	point	V(TS), E, R	Evaporative mass flux data can be collected from an evaporation pan, or flux anemometer	Rimmer et al. (2009)
		Time-series comparison	2b	minutes-hourly	point	V(TS), E, R	Comparison against latent heat fluxes derived from energy balance fitting to surface meterological data	Nussboim et al. (2017)
		H2O isotopes	2b	ad hoc	point	V(other)	Fitting isotopic data can help source identification and compute evaporation rates based on deviation of meteoric water line	Stadnyk et al. (2013)
	Velocity	Time-series comparison	1a	hourly-weekly	point; horizontal transect	V(TS), E, R V(XZ), MAE(DXZ),	Use of point ADCP measurements for point scale or cross section
		Variance ellipse	1c	hourly-weekly	point	V(other)	Summary of magnitude and direction of current field that can be compared with point data	Hetland and DiMarco (2012)
		Residual currents	3	weekly-seasonal	surface layer; horizontal transect	V(XY), V(XZ)	Particle trajectories from model simulations can be compared with tracks from drogues and/or drifters released in the field.	Dissanayake et al. (2019)
	Mixing	Mixing intensity	2a	ad hoc	vertical profile	V(other)	Turbulent diffusivities derived from SCAMP data can guide turbulence parameterisation	Rueda and MacIntyre (2010)
		Tracer dilution	2b	ad hoc	surface layer; horizontal transect	V(TS)	Capturing the horizontal and vertical dispersion of a conservative tracer (e.g., rhodamine or chloride) can ensure diffusion is being accurately captured
	Retention characteristics	Water age variation	3	ad hoc	multiple sites	E, R	The use of radioisotopes could be used to correlate simulated water age with observed estimates from geochemical tracers
		Water source apportionment	3	ad hoc	multiple sites	V(other)	Use of conservative tracers indicating water source from specific surface or groundwater inputs or rainfall, e.g., caffeine, radon, etc.
Heat & salt balance
	Temperature or salinity	Time-series comparison	1a	minutes-monthly	point	V(TS), E, R	Data measured from an in situ thermistor or salinity sensor, or ad hoc measurement	Most papers present this
		Frequency spectra	1c	minutes-hourly	point	FFT, WT	Data measured from a thermistor or salinity sensor logging at high frequency	Kara et al. (2012)
		Spatial comparison	1a	daily-monthly	surface layer	V(XY), MAE(DXY), d2	Satellite acquired temp data (e.g., LANDSAT, MODIS etc) compared pixel for pixel with simulation. Model or data may require averaging to ensure spatial resolutions match	Spillman et al. (2007) Menesguen et al. (2007)
		Spatial variability	1c	daily-monthly	surface layer	DF	Compares distribution and range of T or S variation within the simulated domain without conducting pixel by pixel comparison
		Spatial patchiness	1c	daily-monthly	surface layer	CCF	Can assess similarity in spatial coherence of T or S
		Eddy structure	3	hourly-monthly	water column	V(XY)	Visual comparison of the emergence of complex eddy structures and gyre formation in T or S fields	Holt et al. (2014)
	Temperature	Albedo	2a	ad hoc	surface layer	V(TS), E, R	Models simulating spatiotemporal variability in albedo can validate against estimates computed via upwelling and downwelling pyranometer
		Radiative heat flux	2a	ad hoc	surface layer or benthic layer	V(TS), E, R	Radiative heat flux across the surface of the water or at the sediment-water interface measured using eddy-correlation, microprofiles, IR measurements.
		Benthic perimeter heat exchange	2b	daily-monthly	water column	V(other)	Rate of change of bottom (hyoplimnion) temperature	Salmon et al. (2017)
Stratification
	Temperature, salinity, density	Depth comparison	1a	minutes-monthly	vertical profile (continuous) or multiple depths (discrete)	V(TZ), R, d2 MAE(DTZ),	TZ error contour plot highlights errors in thermocline or pycnocline depth, by comparing interpolated observation and model profiles over time	Menesguen et al. (2007) Missaghi and Hondzo (2010)
		Duration of stratification	1b	hourly-seasonal	water column	BIAS, R, SR, DF	Capturing the total length of time a waterbody experiences stratification can be useful for understanding water quality and/or the impacts of climate change	Frassl et al. (2018)
		Date of water column mixing/over turn	1b	hourly-monthly	water column	BIAS, R, SR, DF	Capturing the specific date of water column overturn may be important when forecasting water quality in reservoirs, for example.
		Lateral gradient	1b	hourly-daily	horizontal transect	V(XZ), MAE(DXZ)	Comparison on lateral gradient in stratification can be used to diagnose model performance in capturing density currents associated with differential surface forcing or boundary inputs	Woodward et al. (2016)
		Internal wave frequency spectra	1c	minutes-hourly	water column	FFT, WT, WC	Comparison of frequency spectra to demonstrate wave periods and modes are being reporeduced	Hodges et al. (2000)
	Velocity	Depth comparison	1a	minutes-hourly	vertical profile	V(TZ), MAE(DTZ)	TZ error contour plot highlights mixing errors, by comparing ADCP data and modelled velocity profiles over time
	Layer structure	Surface mixed-layer depth	1b	daily-monthly	water column	V(TS), E, R	Capturing the mixed layer depth can aid in diagnosing mixing and heat balance problems	Bruce et al. (2018) Steyn and Oke (1982) Acreman and Jeffery (2007) Bayer et al. (2013)
		Metalimnion thickness	1b	daily-monthly	water column	V(TS), E, R	As above, the thickness of the thermocline (or pycnocline) region may assist in validating mixing in lake or ocean models
		Bottom vs surface difference	1b	daily-monthly	2 layer	V(TS), E, R, V(TX)	Time-distance contour plot highlights errors in stratification horizontally, e.g., for assessing seasonal salt-wedge propagation in an estuary	Huang et al. (2018)
	Layer stability	Richardson (Ri) number	1b	daily-monthly	2 layer	V(TS)	The (bulk) Richardson number can be estimated from surface and bottom densities and velocities, to give a quantitative measure of the buoyancy vs inertia forces controlling layer stability
		Schmidt stability	1b	daily-monthly	water column	V(TS), E, R	As above, the Schmidt stability parameter is useful for diagnosing the strength of lake stratification	Bruce et al. (2018)
Bottom morphometry & sediment transport a
	Bottom stress	Time-series comparison	1b	minutes-hourly	point	V(TS), E, R	Stress derived from velocity profile measurements can be used to validate model the bottom stress impacting the rate of resuspension
		Wave attenuation	2b	ad hoc	multiple points	V(other)	Wave driven resuspension is important in shallow systems and model validation could consider wave attenuation with depth and depending on the character of the benthic substrate	Chen et al. (2007)
	Sediment movement	Resuspension rate	2a	ad hoc	point	V(TS), R	In situ experiments measuring resuspension rate can be compared under different hydrodynamic conditions to validate model rates
		Rate of accumulation or erosion of benthic sediments	2b	monthly-decadal	point	V(TS)	For models simulating the change in bottom depth due to sedimentation or erosion, the relative rate of changed in depth measured using hydro-acoustic methods can be used to validate models
		Variation in particle size distribution	3	ad hoc	multiple points	V(other)	Spatial differences in particle size composition of bottom sediment can be used to validate areas of differential deposition rates between particle size classes.
		Spatial changes in bathymetry	3	seasonal-decadal	bottom layer	V(XY), MAE(DXY), d2	Can be used to compare model performance capturing spatial patterns in areas of net accumulation and erosion.
		Wave length, height in sediment undulations	3	ad hoc	bottom layer	V(XY)	Complex patterns that emerge in fine-scale simulations of hydrodynamics and bottom sediment movement	Sun et al. (2010)