feat(example): model-vs-obs surface temperature diagnostic and 5-minute quickstart#769
Merged
Conversation
…minute quickstart Add a GlobalMeanSurfaceTemperatureBias diagnostic to the example provider that compares modelled global-mean surface temperature against HadISST-1-1 observations, exposing the model, reference and bias series plus rmse/mean-bias metrics. Add a curated 'quickstart' data registry holding the single reference file it needs, and a single-page five-minute quickstart guide that installs into a virtual environment, fetches a small dataset, runs the diagnostics and inspects results.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a new ChangesGlobal mean surface temperature bias and quick start
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
packages/climate-ref-example/tests/unit/test_surface_temperature.py (1)
111-167: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPlease add one runnable CMIP7 case.
The implementation now has a separate CMIP7 requirement group and a CMIP7 branch in
_get_model_source_type(), but the executable tests added here still only drive CMIP6 inputs. A smalldefinition_factory(cmip7=...)case would stop that path from regressing silently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: dd057352-6082-4ab9-8994-84a0332d569f
📒 Files selected for processing (11)
changelog/769.feature.mddocs/getting-started/quickstart.mdmkdocs.ymlpackages/climate-ref-example/src/climate_ref_example/__init__.pypackages/climate-ref-example/src/climate_ref_example/surface_temperature.pypackages/climate-ref-example/tests/unit/test_provider.pypackages/climate-ref-example/tests/unit/test_surface_temperature.pypackages/climate-ref/src/climate_ref/__init__.pypackages/climate-ref/src/climate_ref/dataset_registry/quickstart.txtpackages/climate-ref/tests/unit/test_solve_regression/test_solve_regression_global_mean_surface_temperature_bias_.ymlpackages/climate-ref/tests/unit/test_solver/test_solve_metrics.yml
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
Close xarray datasets via context managers in the surface-temperature diagnostic, raise the climate-ref-example xarray floor to >=2025.1.0 (the public xarray.coders.CFDatetimeCoder API), and add a runnable CMIP7 end-to-end test case.
Mint native baselines for the global-mean-surface-temperature-bias diagnostic (default CMIP6 and cmip7 test cases) against HadISST-1-1, recording the committed CMEC bundles and manifests. Native blobs are stored in the ref-baselines-public object store; replay verified both cases match.
Switch the surface-temperature-bias diagnostic from global ts to ocean tos, area-weighted by areacello, for a like-for-like comparison against HadISST-1-1 SST (units aligned to degC). This collapses the previous ~2.7 K coverage-mismatch artefact to a physically sensible ~0.5 K bias. Add two figures (model vs reference series, and the bias over time) to the output bundle, and re-mint the regression baselines.
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2307cee4-62e9-4ed6-aa14-89393527705b
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (14)
changelog/769.feature.mddocs/getting-started/quickstart.mdpackages/climate-ref-example/pyproject.tomlpackages/climate-ref-example/src/climate_ref_example/surface_temperature.pypackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/catalog.yamlpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/manifest.jsonpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/regression/diagnostic.jsonpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/regression/output.jsonpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/catalog.yamlpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/manifest.jsonpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/regression/diagnostic.jsonpackages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/regression/output.jsonpackages/climate-ref-example/tests/unit/test_surface_temperature.pypackages/climate-ref/tests/unit/test_solver/test_solve_metrics.yml
✅ Files skipped from review due to trivial changes (8)
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/regression/diagnostic.json
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/manifest.json
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/manifest.json
- changelog/769.feature.md
- docs/getting-started/quickstart.md
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/regression/diagnostic.json
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/catalog.yaml
- packages/climate-ref-example/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/default/regression/output.json
- packages/climate-ref-example/tests/test-data/global-mean-surface-temperature-bias/cmip7/regression/output.json
…ion guard Rename the diagnostic slug to global-sst-bias (and the committed baseline and solver-regression snapshot to match). Remove the latest_version_files helper: the catalog/solver already returns only the latest dataset version, so the per-execution guard is redundant. The diagnostic now passes the collection paths directly.
Emit the annual model, reference and bias SST series from global-sst-bias as SeriesMetricValue values so they are ingested into the REF database and queryable alongside the scalar metrics, not just available inside the NetCDF. Re-mint the baselines to capture the populated series.json.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a five-minute quick start that gets a new user from install to diagnostic results without building any provider environments or downloading the full reference archive.
Two parts:
New example diagnostic
global-mean-surface-temperature-bias(GlobalMeanSurfaceTemperatureBias). It compares modelled annual global-mean surface temperature (ts) against the HadISST-1-1 observational record, using cosine-of-latitude weighting (noareacellaneeded). The output NetCDF exposes the model series, the reference series, and the bias, and the metric bundle reportsrmseandmean-bias. It supports both CMIP6 and CMIP7 model data via an AND-grouped(model, obs4MIPs reference)requirement. Alatest_version_files()guard ensures that when more than one reference version is present (the full obs4REF archive ships two HadISST-1-1tsversions) only the latest is used, rather than combining overlapping time ranges.Single-page quick start (
docs/getting-started/quickstart.md, linked in the nav as "Quick Start"). It installsclimate-refand the pure-Pythonclimate-ref-exampleprovider into a virtual environment, configures withref config init, fetches the curated sample data plus a single surface-temperature observation from a newquickstartdata registry (a one-file subset of obs4REF, avoiding the multi-gigabyte download), ingests, runs the diagnostics withref solve --provider example, and inspects the results.Stacked on #768 because the quick start uses
ref config init. Rebase ontomainonce #768 merges.Checklist
Please confirm that this pull request has done the following:
changelog/Summary by CodeRabbit