How to compare original and clustered profiles in the latest version?

[lienertajun-glitch]

Hallo! In der älteren Version von Clustering gab es eine Funktion, um die originalen Profile mit den Profilen nach dem Clustering zu vergleichen.(fs_clustered.clustering.plot.compare()) Diese wurde nun entfernt. Wie lässt sich diese Funktion in der neuesten Version umsetzen? Könnten Sie bitte ein Beispiel für Fall 1 geben (d. h. vor der Serialisierung clustering.aggregation_result verwenden)?

[FBumann]

Die neueste version lagert das clustering an ein eigenes package aus, damit flixopt besser maintanbar ist. (https://tsam-xarray.readthedocs.io/en/latest/)

Bitte gib mir ein miniales codebeispiel as <7.0.0. Ich übersetze es dir in 7.0.0

[FBumann]

In 7.x, clustering delegates to tsam_xarray, and the original vs clustered data lives on fs_clustered.clustering.aggregation_result (available before saving). Both profiles are on the original time axis and cover all clustered variables. flixopt bundles the .plotly accessor (xarray_plotly), so a faceted compare plot is a one-liner — stack original/clustered onto a profile dim, then plot:

import xarray as xr

agg = fs_clustered.clustering.aggregation_result
(
    xr.Dataset({'original': agg.original, 'clustered': agg.reconstructed})
    .to_dataarray(dim='profile')
    .plotly.line(x='time', color='profile', facet_row='variable')
    .update_yaxes(matches=None)   # variables have different scales
)

Column names: list(agg.original['variable'].values). Accuracy: agg.accuracy (weighted_rmse, per-variable agg.accuracy.rmse). One variable: agg.original.sel(variable=...) (and the same on agg.reconstructed).

A couple of notes:

• Pre-serialization only. aggregation_result holds the original data; it is not persisted by to_netcdf() / to_json(). Access it before saving, or rebuild via transform.apply_clustering(...) after loading.
• Periods/scenarios come through as extra dims — add e.g. facet_col='scenario'. On this raw object the period dim is named _period (not period).
• Prefer vanilla plotly? agg.original.sel(variable=v) / agg.reconstructed.sel(variable=v) are plain DataArrays you can hand to px.line however you like.

Coming soon (PR #734)

The _period quirk goes away — #734 adds a tidy clustering.compare() (all variables, normal dim names), so it's just:

(
    fs_clustered.clustering.compare()   # all variables; subset via .sel(variable=...)
    .to_dataarray(dim='profile')
    .plotly.line(x='time', color='profile', facet_row='variable')
    .update_yaxes(matches=None)
)

We kept plotting in your hands (rather than re-adding a plot.compare()), since compare() returns plot-ready tidy data.

[FBumann]

Please verify if this resolves your question

[lienertajun-glitch]

In 7.x, clustering delegates to tsam_xarray, and the original vs clustered data lives on fs_clustered.clustering.aggregation_result (available before saving). Both profiles are on the original time axis and cover all clustered variables. flixopt bundles the .plotly accessor (xarray_plotly), so a faceted compare plot is a one-liner — stack original/clustered onto a profile dim, then plot:

import xarray as xr

agg = fs_clustered.clustering.aggregation_result
(
    xr.Dataset({'original': agg.original, 'clustered': agg.reconstructed})
    .to_dataarray(dim='profile')
    .plotly.line(x='time', color='profile', facet_row='variable')
    .update_yaxes(matches=None)   # variables have different scales
)

Column names: list(agg.original['variable'].values). Accuracy: agg.accuracy (weighted_rmse, per-variable agg.accuracy.rmse). One variable: agg.original.sel(variable=...) (and the same on agg.reconstructed).

A couple of notes:

• Pre-serialization only. aggregation_result holds the original data; it is not persisted by to_netcdf() / to_json(). Access it before saving, or rebuild via transform.apply_clustering(...) after loading.
• Periods/scenarios come through as extra dims — add e.g. facet_col='scenario'. On this raw object the period dim is named _period (not period).
• Prefer vanilla plotly? agg.original.sel(variable=v) / agg.reconstructed.sel(variable=v) are plain DataArrays you can hand to px.line however you like.

Coming soon (PR #734)

The _period quirk goes away — #734 adds a tidy clustering.compare() (all variables, normal dim names), so it's just:

(
    fs_clustered.clustering.compare()   # all variables; subset via .sel(variable=...)
    .to_dataarray(dim='profile')
    .plotly.line(x='time', color='profile', facet_row='variable')
    .update_yaxes(matches=None)
)

We kept plotting in your hands (rather than re-adding a plot.compare()), since compare() returns plot-ready tidy data.

Hallo,
mithilfe Ihrer Antwort habe ich meinen Code angepasst, um die Diagramme für alle Profile automatisch zu erstellen.
Das Rendering aller Profile (original und clustered) sieht wie folgt aus:

agg = fs_clustered.clustering.aggregation_result
(
xr.Dataset({'original': agg.original, 'clustered': agg.reconstructed})
.to_dataarray(dim='profile')
.plotly.line(x='time', color='profile', facet_row='variable')
.update_yaxes(matches=None) # variables have different scales
.show()
)

Nach dem Auswählen spezifischer Variablen (sel(variable=target_vars)) sieht das Ergebnis deutlich besser aus:

agg = fs_clustered.clustering.aggregation_result
target_vars = ['RH_Bedarf_IST(Q_th)|fixed_relative_profile', 'TWE_Bedarf(Q_th)|fixed_relative_profile']
(
xr.Dataset({'original': agg.original.sel(variable=target_vars), 'clustered': agg.reconstructed.sel(variable=target_vars)})
.to_dataarray(dim='profile')
.plotly.line(x='time', color='profile', facet_row='variable')
.update_yaxes(matches=None)
.show()
)
Damit ist mein Problem im Wesentlichen gelöst. Im Vergleich zur alten Version (fs_clustered.clustering.plot.compare()) wird das Bild bei vielen Profilen jedoch schnell sehr unübersichtlich/überladen...

Vielen Dank für Ihre schnelle Hilfe!

[FBumann]

Gerne.
Für mehr plotting optionen schau in die api reference von plotly.express.line() oder hier: https://xarray-plotly.readthedocs.io/en/v0.0.17/examples/dimensions/#animation

Wahrscheinlich line(..., animation_frame='variable') statt facet_row='variable'