You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Configured transforms are now applied during scoring (#40).
scores.csv is now emitted in wide format, with transformed-scale metrics as <metric>__<label>-suffixed columns (e.g. wis__log). Setting append: false emits only the suffixed columns.
Transform-invariant metrics (interval_coverage_<n> and bias) are now always reported on a single, un-suffixed scale in scores.csv and predevals-options.json, even when a transform is configured. Both are unchanged by any strictly monotonic transform (interval containment and the sign of forecast-vs-observation error are rank-based), so the transformed-scale column was a duplicate of the natural-scale column. Matches the invariance set documented in hubverse-org/hubDocs#464 (#63).
Existing v1.0.1 configs continue to validate against v1.1.0 without changes.
Added generate_predevals_options(), which assembles the contents of the predevals-options.json file used to initialise the predevals dashboard. It returns the validated config with each target's metrics expanded to the columns present in scores.csv (relative-skill metrics, plus transformed-scale <metric>__<label> metrics when a transform applies) and a resolved transform block attached (#41, closes #4).
generate_eval_data() now discovers oracle output from hub_path via hubData::connect_target_oracle_output() when oracle_output is not supplied; the argument remains supported for back-compat (#51).
Bug Fixes
generate_eval_data() no longer fails on ordinal pmf targets that request rps. The ordinal level order is now read from the hub's tasks.json and forwarded to hubEvals::score_model_out() so scoringutils dispatches the data as ordinal (#48).
read_predevals_config() now warns when an ordinal-only pmf metric (e.g. rps) is requested against a pre-v4 tasks-schema (where output_type_id is split across required/optional), and errors if the hub's pmf output_type_id$optional is non-empty (#48).