Skip to content

Add support for vertical mod cleaned#609

Merged
otzi5300 merged 10 commits intovertical-skillfrom
add_support_for_vertical_mod_cleaned
Mar 17, 2026
Merged

Add support for vertical mod cleaned#609
otzi5300 merged 10 commits intovertical-skillfrom
add_support_for_vertical_mod_cleaned

Conversation

@otzi5300
Copy link
Contributor

@otzi5300 otzi5300 commented Mar 16, 2026

#close 596

Step 2 of 4 in implementation of vertical skills

General idea of vertical skill is to store data for model and obs in 1d (see below). Depth information is stored in auxilary coordinate z. Befinit of this compared to storing in 2d:

  • Similar to current data structures and current matching workflow can be used
  • Allowing varying z coordinates for obs and sigma layered models

Steps:

  1. BRANCHED FROM Support vertical obs
  2. Support vertical model items
  3. Matching to get comparer
  4. plotting/skill etc on vertical

Support vertical mod per commit

(see vertical skill notebook for more info)

Bugfix
Using a dataset to DfsuModelResults did not work. Sel_items and items was not set. Fixed.

VerticalModelResult class
Stores timeseries data for vertical profiles.

Support extraction of dfsu to VerticalModelResults

  • add _extract_vertical() in dfsu.py
  • Type-hints fro VerticlModelResults in comparer and spatial

Updated notebook vertical_skill.ipynb with examples for review process

Formatting
Reformatting TaylorPoint-line in comparer_plotter.py (Ruff wanted this)

Removing outcommented coded and fixing comments

Testdata

  • dfs0 file for creating VerticalModelResults

Tests
Testing various routes for creating model, edge cases and extracting VerticalModelResults from dfsu

Not sure why ruff want to change this line all of the sudden. To keep Ruff from complaining i will commit it.
* DfsuModelResults would fail if Dataset is passed as input since sel_items and item not set when extracting variables
@otzi5300 otzi5300 linked an issue Mar 16, 2026 that may be closed by this pull request
Copy link
Collaborator

@ryan-kipawa ryan-kipawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, super easy to follow! Looking forward to the upcoming PRs :)

# Test basic open for different formats
# ================
# dataframe input
def test_vertical_model_result_from_dataframe(self, vertical_model_df):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These three tests could be combined into one by parameterizing the input source (using pytest.mark.parameterize).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, fixed

keep_duplicates : (str, bool), optional
Strategy for handling duplicate timestamps (wraps xarray.Dataset.drop_duplicates)
"first" to keep first occurrence, "last" to keep last occurrence,
False to drop all duplicates, "offset" to add milliseconds to
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to use a string instead of False, so that they're all string literals with nice completion?

Copy link
Contributor Author

@otzi5300 otzi5300 Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is a good idea but other observations and models (point, track) uses "first" "last" false.

my suggestion is to keep the implementation of vertical as it is but open an separate issue to deal with this for all types of observations and models. Also this type of argument is consistent with pandas https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.duplicated.html


assert mr.equals(mr2)
assert mr2.gtype == mr.gtype
assert mr2.name == mr.name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you update one of these on mr, does mr2 remain in tact or is it also changed? I've encountered something like this before in modelskill, and it's not always obvious that it's happening, and caused some extra effort to sort out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have added test that changing the name in original does not alter name in the roundtripped

ntimes = len(np.unique(vmr.data.time.values))
n_layers = int(len(vmr.data.z.values) / ntimes)

assert n_layers == n_layers_expected
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asserts correct shape. Should we also have an assertion that the correct values are being extracted? Maybe not all of them, but first and last.

Copy link
Contributor Author

@otzi5300 otzi5300 Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test added

@@ -0,0 +1,77 @@
from __future__ import annotations
from typing import Any, Literal, Optional, Sequence
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional is old school typing. It's preferred to use type | None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed but now it is not fully consistent with the rest of the code. Maybe we should consider changing the rest of the code as well

@otzi5300 otzi5300 removed the request for review from ecomodeller March 17, 2026 09:06
@otzi5300 otzi5300 merged commit ea4c9e3 into vertical-skill Mar 17, 2026
Copy link
Member

@ecomodeller ecomodeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mikeio.read -> Dfsu3d.read already supports reading a column with x,y, see this notebook
https://github.com/DHI/mikeio/blob/main/notebooks/Dfsu%20-%203D%20sigma-z.ipynb

elif isinstance(self.data, mikeio.dfsu.Dfsu3D):
# FIXME: open with specific element instead...make sure mikeio fix is in place before
# ds_column = self.data.read(elements=elemids) # when bug is fixed in mikeio
ds_column = self.data.read(items=self.sel_items.all).sel(x=x, y=y)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not self.data.read(..., x=x,y=y)? Skip the .sel


if not self._in_domain(x, y):
raise ValueError(
f"PointObservation '{observation.name}' (x={x}, y={y}) outside model domain!"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Point?

aligned = pmr.align(observation, max_gap=max_model_gap)
case VerticalModelResult(), VerticalObservation():
raise NotImplementedError("Vertical matching not implemented yet!")
# aligned = vmr.align(observation, max_gap=max_model_gap)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add TODO to make this findable?

time_flat = np.repeat(ds_column.time, n_layers)
z_flat = layer_boundaries.flatten() # Flatten z-coordinates
item_values_1d = ds_column[item_name].to_numpy().flatten() # Flatten item?
# aux_items_flat = {aux_item: ds_column[aux_item].to_numpy().flatten() for aux_item in self.sel_items.aux}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the commented code stay or be deleted?

f"PointObservation '{observation.name}' (x={x}, y={y}) outside model domain!"
)

elemids = self.data.geometry.find_index(x=x, y=y)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary

@otzi5300 otzi5300 deleted the add_support_for_vertical_mod_cleaned branch March 17, 2026 09:13
@otzi5300 otzi5300 linked an issue Mar 17, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Matching VerticalObservation and ColumnModelResults Add ModelResult for vertical column

3 participants