Skip to content

fix: drop stored weights on apply() when columns are absent#100

Merged
FBumann merged 1 commit into
mainfrom
fix/apply-drop-missing-weights
Jul 22, 2026
Merged

fix: drop stored weights on apply() when columns are absent#100
FBumann merged 1 commit into
mainfrom
fix/apply-drop-missing-weights

Conversation

@FBumann

@FBumann FBumann commented Jul 22, 2026

Copy link
Copy Markdown
Owner

What

ClusteringResult.apply() (transfer path) now drops the stored clustering weights — with a UserWarning — when a weighted column is absent from the new data, instead of propagating tsam's ValueError: Weight columns not found in data.

Why

At apply time the cluster assignments and centers are already fixed, so stored weights cannot change which periods are selected. Their only remaining effect is on the weighted accuracy metrics. tsam nonetheless validates the stored weight columns against the new data and hard-fails, which blocks a legitimate use case: transferring a clustering built on a subset to a differently-composed dataset.

Verified empirically — stripping the weights leaves cluster_representatives and reconstructed byte-identical; only weighted_rmse/weighted_mae change (and those weights referenced columns not even present in the new data).

This is the symmetric counterpart of the already-fixed tsam #276 (extra columns) — here the new data is missing a weighted column.

Changes

  • _drop_missing_weights() in _clustering.py, called from _apply_single(). Drops all weights when any are missing (an all-or-nothing choice — a partial re-weighting would silently diverge from the source intent, and the core outputs don't move either way). No-ops when all weight columns are present, so same-composition transfers keep their weighted metrics and emit no warning.
  • TestApplyWeightTransfer: warns-and-succeeds, no-warning-when-present, plus a canary (test_upstream_still_rejects_missing_weight_columns) that asserts raw tsam still raises. When it flips red, the upstream fix has landed and this workaround can be removed.

Upstream

Tracking: FZJ-IEK3-VSA/tsam#396

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Applying a weighted clustering to data missing previously used weight columns no longer fails.
    • A warning is shown when weight columns are unavailable, while reconstructed results remain available.
    • Applications using data with all required weight columns continue without warnings.

tsam's ClusteringResult.apply() hard-fails when a stored weight column
is missing from the new data, blocking transfer of a clustering to a
differently-composed dataset. At apply time cluster assignments and
centers are already fixed, so weights cannot change selection — their
only remaining effect is on the weighted accuracy metrics. When a
weighted column is absent we now drop all weights and warn instead of
propagating the ValueError.

A canary test asserts raw tsam still raises; it flips red once the
upstream fix (FZJ-IEK3-VSA/tsam#396) lands, signalling this workaround
can be removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The clustering application path now detects missing stored weight columns, emits a warning, and applies the clustering without weights. Tests cover missing-column handling, complete-column transfers, and unchanged upstream tsam behavior.

Changes

Weighted clustering transfer

Layer / File(s) Summary
Handle missing weights during clustering application
src/tsam_xarray/_clustering.py, test/test_aggregate.py
The application path removes unavailable weights with a warning before calling cr.apply(...); tests verify reconstruction, warning behavior, complete weighted inputs, and upstream ValueError behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: dropping stored weights during apply() when referenced columns are missing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/apply-drop-missing-weights

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/tsam_xarray/_clustering.py (1)

566-578: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the required Google-style docstring sections.

Add applicable Args: and Returns: sections for cr, df, and the returned clustering result.

As per coding guidelines, use Google-style docstrings with Args:, Returns:, Raises:, Attributes: sections (not NumPy style).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/tsam_xarray/_clustering.py` around lines 566 - 578, Add Google-style
`Args:` entries documenting the `cr` and `df` parameters, and a `Returns:` entry
documenting the clustering result returned by this workaround. Keep the existing
behavior and explanatory text unchanged, and use the surrounding function’s
actual parameter and return types or descriptions.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/tsam_xarray/_clustering.py`:
- Around line 566-578: Add Google-style `Args:` entries documenting the `cr` and
`df` parameters, and a `Returns:` entry documenting the clustering result
returned by this workaround. Keep the existing behavior and explanatory text
unchanged, and use the surrounding function’s actual parameter and return types
or descriptions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ec54c565-9493-4a29-a582-4e6694fab953

📥 Commits

Reviewing files that changed from the base of the PR and between ec799a5 and aa78238.

📒 Files selected for processing (2)
  • src/tsam_xarray/_clustering.py
  • test/test_aggregate.py

@FBumann
FBumann merged commit 5616d1c into main Jul 22, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant