Motivation
Currently every model's `verify_physics_fn` only checks STATE RANGES and FINITENESS:
```python
return {
'W_in_0_1': ..., 'Zt_in_0_A': ..., 'a_nonneg': ..., 'all_finite': ...,
}
```
This is necessary but not sufficient. A model can pass these checks while producing biologically implausible data (see #5 — SWAT Set A's Zt amplitude never reaches `c_tilde + delta_c` so deep-sleep happens only via stochastic noise floor; the simulated truth exhibits unrealistic sleep architecture).
These model-design issues should be surfaced at the model layer (this repo), not 1-2 stages downstream (psim or SMC²). Wasted GPU time on the SMC² port for SWAT was traceable to two such issues plus a framework bug.
Proposal
Extend each model's `verify_physics_fn` to also return informational biological-realism metrics. These are NOT gating (they don't fail the CLI's `PASS: physics verification` line) but they SHOULD be surfaced clearly in the run output so the modeller spots problems at simulation time.
Convention: any key ending in `_realistic` is a string `"yes"/"no"` (informational, not gating). Numeric metrics (e.g. `sleep_fraction: 0.21`) are also informational. Only `_ok` / `in*` style booleans gate.
SWAT-specific (Set A as worked example)
`models/swat/simulation.py:verify_physics`:
```python
def verify_physics(trajectory, t_grid, params):
W = trajectory[:, 0]; Zt = trajectory[:, 1]; a = trajectory[:, 2]; T = trajectory[:, 3]
# Existing range / finite checks
res = {
'W_in_0_1': bool((W >= 0).all() and (W <= 1).all()),
'Zt_in_0_A': bool((Zt >= 0).all() and (Zt <= A_SCALE).all()),
'a_nonneg': bool((a >= 0).all()),
'T_nonneg': bool((T >= 0).all()),
'T_bounded': bool(T.max() < 100.0),
'all_finite': bool(np.all(np.isfinite(trajectory))),
}
# NEW: informational realism metrics
c1 = params['c_tilde']
c2 = c1 + params['delta_c']
sleep_frac = float((Zt > c1).mean())
deep_sleep_frac = float((Zt > c2).mean())
Zt_max = float(Zt.max())
res['sleep_fraction'] = sleep_frac
res['deep_sleep_fraction'] = deep_sleep_frac
res['Zt_max'] = Zt_max
res['T_range'] = float(T.max() - T.min())
res['T_mean'] = float(T.mean())
# Realism flags (string-typed; informational only)
res['sleep_fraction_realistic'] = (
\"yes\" if 0.25 <= sleep_frac <= 0.40 else \"no\")
res['Zt_reaches_deep_threshold_realistic'] = (
\"yes\" if Zt_max > c2 else \"no\")
res['deep_sleep_fraction_realistic'] = (
\"yes\" if 0.05 <= deep_sleep_frac <= 0.15 else \"no\")
return res
```
When SWAT Set A is simulated by `run_simulator.py --model models.swat.simulation.SWAT_MODEL --param-set A`, the output would now show:
```
sleep_fraction 0.057
deep_sleep_fraction 0.000
Zt_max 4.30
sleep_fraction_realistic no <- modeller spots this
Zt_reaches_deep_threshold_realistic no
deep_sleep_fraction_realistic no <- and this
all_finite PASS
PASS: physics verification
```
`PASS` still fires on the gating booleans (which are unchanged); the informational `no` flags clearly surface the realism concerns.
Other models
Each model's verify_physics_fn should be similarly extended:
- `fsa_real_obs` / `fsa_high_res`: realistic resting-HR range, fitness-buildup cycles
- `sleep_wake` / `sleep_wake_20p`: sleep architecture metrics
- `bistable_controlled`: state-switching frequency
Per-model definition; `how_to_add_a_new_model/03_testing_and_docs.md` should be updated to include realism checks as part of the model-developer responsibility.
Acceptance criteria
Cross-reference
- #5 (SWAT Set A Zt amplitude — the motivating case)
- psim follow-ups: per-model diagnostic plot at packaging + realism-warning surfacing in validate_simrun (separate issues to be opened)
Motivation
Currently every model's `verify_physics_fn` only checks STATE RANGES and FINITENESS:
```python
return {
'W_in_0_1': ..., 'Zt_in_0_A': ..., 'a_nonneg': ..., 'all_finite': ...,
}
```
This is necessary but not sufficient. A model can pass these checks while producing biologically implausible data (see #5 — SWAT Set A's Zt amplitude never reaches `c_tilde + delta_c` so deep-sleep happens only via stochastic noise floor; the simulated truth exhibits unrealistic sleep architecture).
These model-design issues should be surfaced at the model layer (this repo), not 1-2 stages downstream (psim or SMC²). Wasted GPU time on the SMC² port for SWAT was traceable to two such issues plus a framework bug.
Proposal
Extend each model's `verify_physics_fn` to also return informational biological-realism metrics. These are NOT gating (they don't fail the CLI's `PASS: physics verification` line) but they SHOULD be surfaced clearly in the run output so the modeller spots problems at simulation time.
Convention: any key ending in `_realistic` is a string `"yes"/"no"` (informational, not gating). Numeric metrics (e.g. `sleep_fraction: 0.21`) are also informational. Only `_ok` / `in*` style booleans gate.
SWAT-specific (Set A as worked example)
`models/swat/simulation.py:verify_physics`:
```python
def verify_physics(trajectory, t_grid, params):
W = trajectory[:, 0]; Zt = trajectory[:, 1]; a = trajectory[:, 2]; T = trajectory[:, 3]
```
When SWAT Set A is simulated by `run_simulator.py --model models.swat.simulation.SWAT_MODEL --param-set A`, the output would now show:
```
sleep_fraction 0.057
deep_sleep_fraction 0.000
Zt_max 4.30
sleep_fraction_realistic no <- modeller spots this
Zt_reaches_deep_threshold_realistic no
deep_sleep_fraction_realistic no <- and this
all_finite PASS
PASS: physics verification
```
`PASS` still fires on the gating booleans (which are unchanged); the informational `no` flags clearly surface the realism concerns.
Other models
Each model's verify_physics_fn should be similarly extended:
Per-model definition; `how_to_add_a_new_model/03_testing_and_docs.md` should be updated to include realism checks as part of the model-developer responsibility.
Acceptance criteria
Cross-reference