Install pcmdi_metrics from a fork of 4.2.0 with a chunked variability-modes SVD - #901
Open
lewisjared wants to merge 7 commits into
Open
Install pcmdi_metrics from a fork of 4.2.0 with a chunked variability-modes SVD#901lewisjared wants to merge 7 commits into
lewisjared wants to merge 7 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (34)
🚧 Files skipped from review as they are similar to previous changes (2)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe PMP provider now installs a pinned ChangesPMP diagnostics and reference data
Merge Risk: 🟡 Moderate · up to This update installs a PMP fork over the existing conda package to improve diagnostic performance and refreshes diagnostic outputs. Merge readiness depends on confirming that the overlay preserves the intended dependency set; otherwise ENSO processing may fail at runtime. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 5 files. (30 skipped: 30 unsupported.) ✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
lewisjared
marked this pull request as ready for review
September 3, 2026 05:47
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 26 files with indirect coverage changes 🚀 New features to boost your workflow:
|
lewisjared
changed the title
lewisjared
had a problem deploying
to
native-baselines
September 3, 2026 07:58 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Contributor
Author
|
This will invalidate the existing pmp runs so we should bump with #898 |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-pr/regression/diagnostic.json (1)
85-85: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winRe-mint the PMP provenance files with NumPy 2.2.6.
The conda lock file resolves NumPy 2.2.6, but the diagnostic and output provenance files record NumPy 2.0.2. Update all affected annual-cycle provenance fixtures.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Team
Run ID: 1f2b2738-29fc-4662-a18a-c7d5aceeda85
📒 Files selected for processing (59)
changelog/901.fix.mdpackages/climate-ref-pmp/src/climate_ref_pmp/__init__.pypackages/climate-ref-pmp/src/climate_ref_pmp/diagnostics/annual_cycle.pypackages/climate-ref-pmp/src/climate_ref_pmp/diagnostics/enso.pypackages/climate-ref-pmp/src/climate_ref_pmp/diagnostics/variability_modes.pypackages/climate-ref-pmp/src/climate_ref_pmp/requirements/conda-lock.ymlpackages/climate-ref-pmp/src/climate_ref_pmp/requirements/environment.ymlpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-pr/manifest.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-pr/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-pr/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ta/manifest.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ta/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ta/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ts/manifest.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ts/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip6-ts/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip7-ts/manifest.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip7-ts/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/annual-cycle/cmip7-ts/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip6/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nam/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nam/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nam/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nam/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nao/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nao/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nao/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-nao/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npgo/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npgo/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npgo/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npgo/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npo/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npo/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npo/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-npo/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pdo/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pdo/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pdo/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pdo/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pna/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pna/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pna/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-pna/cmip7/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-sam/cmip6/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-sam/cmip6/regression/diagnostic.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-sam/cmip7/manifest.jsonpackages/climate-ref-pmp/tests/test-data/extratropical-modes-of-variability-sam/cmip7/regression/diagnostic.json
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/climate-ref-pmp/src/climate_ref_pmp/init.py
- changelog/901.fix.md
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
| "osAccess": false, | ||
| "packages": { | ||
| "PMP": "3.9.2", | ||
| "PMP": "4.2.0", |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🤖 get_repo_knowledge executed:
get_repo_knowledge Climate-REF/climate-ref /tmp/coderabbit-repo-knowledge/climate-ref-climate-ref-2aca5c04/learnings
Length of output: 4627
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed files and focused diff ---'
git diff --stat -- packages/climate-ref-pmp
git diff -- packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json
printf '%s\n' '--- NumPy and PMP version references in tracked package/config files ---'
rg -n -i --glob '!**/node_modules/**' --glob '!**/dist/**' \
'numpy|NumPy|PMP' packages/climate-ref-pmp .github pyproject.toml setup.cfg setup.py tox.ini 2>/dev/null || true
printf '%s\n' '--- focused provenance blocks ---'
for f in \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json
do
echo "--- $f"
sed -n '55,70p;200,214p;266,280p' "$f" 2>/dev/null || true
doneRepository: Climate-REF/climate-ref
Length of output: 50380
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed paths ---'
git diff --name-only
printf '%s\n' '--- focused fixture diff ---'
git diff --unified=3 -- \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json
printf '%s\n' '--- changed NumPy references ---'
git diff --unified=0 -- . \
':(exclude)packages/climate-ref-pmp/tests/unit' \
':(exclude)packages/climate-ref-pmp/tests/test-data' |
rg -n -i -C 2 'numpy|NumPy|2\.2\.6|2\.0\.2' || true
printf '%s\n' '--- tracked configuration references ---'
rg -n -i --glob '*.toml' --glob '*.yaml' --glob '*.yml' --glob '*.lock' --glob '*.cfg' \
--glob '!packages/climate-ref-pmp/tests/**' \
'numpy|NumPy|2\.2\.6|2\.0\.2' . || true
printf '%s\n' '--- exact fixture provenance values ---'
python3 - <<'PY'
import json
from pathlib import Path
files = [
"packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json",
]
for name in files:
data = json.loads(Path(name).read_text())
found = []
def walk(value, path="$"):
if isinstance(value, dict):
for key, child in value.items():
if key.lower() in {"numpy", "numpy_version"}:
found.append((f"{path}.{key}", child))
walk(child, f"{path}.{key}")
elif isinstance(value, list):
for i, child in enumerate(value):
walk(child, f"{path}[{i}]")
walk(data)
print(name, found)
PYRepository: Climate-REF/climate-ref
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- fixture provenance values ---'
python3 - <<'PY'
import json
from pathlib import Path
files = [
"packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json",
"packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json",
]
for name in files:
data = json.loads(Path(name).read_text())
matches = []
def walk(value, path="$"):
if isinstance(value, dict):
for key, child in value.items():
if "numpy" in key.lower():
matches.append((f"{path}.{key}", child))
walk(child, f"{path}.{key}")
elif isinstance(value, list):
for index, child in enumerate(value):
walk(child, f"{path}[{index}]")
walk(data)
print(name)
for item in matches:
print(" ", item)
PY
printf '%s\n' '--- environment pin ---'
cat -n packages/climate-ref-pmp/src/climate_ref_pmp/requirements/environment.yml
printf '%s\n' '--- locked NumPy package entries ---'
sed -n '6908,6960p' packages/climate-ref-pmp/src/climate_ref_pmp/requirements/conda-lock.yml
printf '%s\n' '--- provenance context ---'
for f in \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json \
packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json
do
echo "--- $f"
rg -n -C 3 '"PMP"|"numpy"|"NumPy"' "$f"
doneRepository: Climate-REF/climate-ref
Length of output: 5897
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- provenance generation and regression contract ---'
rg -n -i -C 3 \
'PROVENANCE|provenance|packages\.numpy|numpy.*version|version.*numpy' \
packages/climate-ref-pmp/src packages/climate-ref-pmp/tests \
-g '*.py' -g '*.json' -g '*.yml' -g '*.yaml' |
head -n 240Repository: Climate-REF/climate-ref
Length of output: 28260
Align regression provenance with NumPy 2.2.6.
The PMP lock resolves NumPy 2.2.6, but all six listed fixtures record 2.0.2. Regenerate the fixtures or update their provenance values.
📍 Affects 6 files
packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/diagnostic.json#L63-L63(this comment)packages/climate-ref-pmp/tests/test-data/enso_proc/cmip7/regression/output.json#L208-L208packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/diagnostic.json#L65-L65packages/climate-ref-pmp/tests/test-data/enso_tel/cmip6/regression/output.json#L274-L274packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/diagnostic.json#L65-L65packages/climate-ref-pmp/tests/test-data/enso_tel/cmip7/regression/output.json#L274-L274
lewisjared
changed the title
Contributor
Author
Second mint failed, holding the conda pin at 3.9.2Mint run 33730826484 bumped the conda environment to
The metric then goes missing from the output bundle, so
Tracked in #906, along with hardening |
…es SVD The fork tracks upstream 4.2.0 and chunks the SVD in the variability modes analysis. This cuts the runtime of the extratropical modes of variability diagnostics by roughly 40x. It goes on with --no-deps over the 3.9.2 conda pin, which cannot move until #906 is resolved.
The chunked SVD in the forked pcmdi_metrics changes the variability modes results, so every PMP baseline has to be re-minted. Bumping all three diagnostics together keeps their versions moving in step.
PMP computes 12 monthly values per statistic in `CalendarMonths`, which the transform step dropped on the way to the scalar bundle. This left `series.json` empty for every annual cycle case. - Extracts one 12 point series per region and statistic, indexed by month number. - Adds a `level` dimension for the 3D pressure-level variables. - Gives the series the same identifying dimensions as the scalars. - Bumps the diagnostic version to 7 and re-mints the four baselines from replay. The native artefacts are unchanged, so the mint reused the stored blobs.
* origin/main: refactor: share one transport between the native and report stores test: cover the report store's error paths fix: keep an unexpected key from tracebacking mid-upload refactor: tidy the report upload after review fix: name the reports bucket ref-baselines-reports fix: report an absent credential instead of a traceback fix: address the review of the report upload feat: publish the baseline diff report and write the PR comment ci: post baseline diff reports from a dedicated workflow fix: measure netcdf differences without losing precision or missingness feat: emphasise the netcdf values that actually moved feat: show the whole netcdf header, with a side by side view chore: rename the changelog fragment to the PR number refactor: tighten the netcdf analysis after the cleanup reviews feat: add netcdf stats to the baseline diff report chore: add a changelog fragment for the mypy 2 bump chore(deps): raise the mypy requirement to match the lockfile chore(deps): bump mistune from 3.3.0 to 3.3.3 chore(deps): update dependency mypy to v2
Contributor
Regression baseline diff22 test case(s) changed against
|
Contributor
Author
|
@lee1043 Can you take a look at the diffs in the comment above? There are more differences in the values than I would have expected, but mostly small. The ta annual cycle @ 850 HPa did have some larger changes. https://reports.baselines.climate-ref.org/901/23f40bd0472e/pmp/annual-cycle/cmip6-ta/index.html |
Contributor
|
@lewisjared thanks for bringing this to my attention and thanks for implementing updated PMP version into REF. It is interesting that the chance is in the annual cycle ta-850 correlation value, while for the other fields correlation remained unchanged. Let me take a look into it and get back to you soon. |
Contributor
|
@lewisjared my initial guess is it might be related to the change in newer numpy regarding how it handles NaN values -- as only 850 hPa field correlation calculation has been substantially changed. I will continue investigating.. |
Contributor
|
Hi @lewisjared , We (PMP team) have updated PMP to v4.2.1 that now includes clarification for this #901 (comment), which was followed up in this PCMDI/pcmdi_metrics#1426. In summary, change in the correlation of ta-850 is resulted from "corrected" calculation for correlation when there are NaN value in the field, so I confirm that was expected change. We updated the format to display up to three decimal places, fixing an issue where values like 1.00 or -1.00 were previously rounded to two decimal places. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Installs
pcmdi_metricsfrom a fork of 4.2.0 that chunks the SVD in the variability modes analysis. This cuts the runtime of the extratropical modes of variability diagnostics by roughly 40x.The fork is pinned to commit
d0a79e5onlewisjared/pcmdi_metrics@fix/variability-modes-dask-svd-memory, two commits ahead of upstream 4.2.0. It follows the pattern the ESMValTool provider already uses: conda installs the released package for its dependencies, then pip installs the fork over the top with--no-deps. The conda pin stays at 3.9.2, so the diagnostics run 4.2.0 code on a 3.9.2 dependency set. That mismatch is deliberate and tracked in #906, because moving the conda pin to 4.2.0 pullsenso_metrics2.0.0, whose CDAT to xarray migration breaksEnsoFbSstTauxand fails bothenso_proccases.Diagnostic.versionis bumped on all three PMP diagnostics, and all 22 baselines are re-minted from run 33721507480.Things worth a close look.
cmip6-tais the interesting one.cor_xywas between 0.86 and 0.97 depending on season and is now roughly 0.996 in every season, which reads as a fixed misalignment rather than drift.tais the only 3D case here.enso_metricsstays at 1.1.5.Summary by CodeRabbit
New Features
Bug Fixes
Tests