Skip to content

v1.2.0

Choose a tag to compare

@annakrystalli annakrystalli released this 20 Jul 12:42
· 25 commits to main since this release
v1.2.0
82b2684

Dependencies

Refreshed renv.lock to pull the latest releases of the hubverse packages the
image wraps: hubPredEvalsData 1.1.1 -> 1.2.0, hubEvals 0.2.0 -> 0.3.1,
hubUtils 1.2.0 -> 1.2.1, and hubData 2.2.1 -> 2.2.2 (#47).

The user-visible changes these bring to the generated dashboard data:

  • predevals-options.json targets now carry target_name and target_units,
    read from the hub's tasks config. The dashboard uses target_name to label
    target menu items (hubverse-org/hubPredEvalsData#21).
  • scores.csv rows are now sorted on the (model_id, by) key before writing,
    so the file is byte-stable across runs. Output can be diffed or
    md5-compared to confirm a no-change situation
    (hubverse-org/hubPredEvalsData#25).
  • Models scored only on a later output type are no longer silently dropped
    from scores.csv. The per-output-type score frames are now merged with a
    full join rather than a left join anchored on the first output type, so
    which models survived no longer depends on the order metrics happen to be
    listed in the config (hubverse-org/hubPredEvalsData#75).
  • Relative-skill scoring no longer aborts the run on single-model input, or
    on disaggregated comparison groups that cannot be compared. Such groups are
    reported as relative skill 1 or NA with a warning, and their absolute
    scores are returned unchanged (hubverse-org/hubEvals#75,
    hubverse-org/hubEvals#135).
  • Scoring on transformed scales with transform_append now returns one row
    per scale instead of silently averaging across scales
    (hubverse-org/hubEvals#122).
  • Oracle output is now read correctly from cloud (S3) hubs whose admin.json
    declares a non-parquet submission format such as csv. The cloud sync always
    writes model-output/ to parquet on S3 regardless of the declared format, so
    such hubs previously came back as an empty connection
    (hubverse-org/hubData#148).

Full Changelog: v1.1.1...v1.2.0