Skip to content

Extracting data directly from edge#637

Open
jpalm3r wants to merge 61 commits intomainfrom
edge_observations
Open

Extracting data directly from edge#637
jpalm3r wants to merge 61 commits intomainfrom
edge_observations

Conversation

@jpalm3r
Copy link
Copy Markdown
Collaborator

@jpalm3r jpalm3r commented Apr 20, 2026

Issue

Network data is stored in nodes or breakpoints and, to access the latter, you need the edge name and the distance within the edge (i.e. chainage). There are variables that are constant in all breakpoints from the same edge (e.g. discharge), thus requiring the chainage is redundant. In addition, in some domains, users typically do not use the chainage at all which creates friction.

This PR introduces the following changes:

Enhanced observation and matching capabilities:

  • Introduced the EdgeObservation type and integrated it throughout the codebase, allowing extraction and matching of model results at arbitrary breakpoints along network edges, not just at named nodes. [1] [2] [3] [4] [5] [6]
  • Updated the NetworkModelResult.extract() method to support both NodeObservation and EdgeObservation, with improved error messages and alias resolution for node and edge identifiers. [1] [2]

Documentation improvements:

  • Expanded the user guide with detailed explanations and examples for selective loading, node and edge observation options, and improved tips for skill assessment workflows. [1] [2] [3]

Example use

obs_q = ms.EdgeObservation(path_to_sensor_data, edge="94l1")
mr_q = NetworkModelResult(network, name="MyModel", item="Discharge")
cc_q = ms.match(obs=obs_q, mod=mr_q)
cc_q.skill()

@jpalm3r jpalm3r requested a review from ryan-kipawa April 22, 2026 09:01
@jpalm3r jpalm3r changed the title Exctracting data directly from edge Extracting data directly from edge Apr 22, 2026
@jpalm3r jpalm3r requested a review from Copilot April 22, 2026 12:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands network-based workflows by introducing edge-based observations and alias/breakpoint resolution, enabling extraction/matching at arbitrary breakpoints along edges (not only named nodes). It also adds selective loading options for large Res1D networks and updates the user guide accordingly.

Changes:

  • Added EdgeObservation and extended NetworkModelResult.extract() to support both node- and edge-based extraction, including alias resolution for string IDs and (edge, distance) breakpoints.
  • Implemented selective loading in Network.from_res1d(nodes=..., reaches=...) to reduce memory usage while preserving full topology.
  • Updated documentation and tests for alias handling and selective loading behavior.

Reviewed changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/test_network.py Adds tests for selective loading and alias/breakpoint resolution.
tests/test_match.py Updates error-message expectation for network matching.
src/modelskill/timeseries/_timeseries.py Extends dataset validation to accept edge / edge+distance coords.
src/modelskill/timeseries/_point.py Adds EdgeCoords / breakpoint parsing and adjusts gtype attribution.
src/modelskill/timeseries/_coords.py Introduces NodeCoords and new EdgeCoords.
src/modelskill/timeseries/init.py Exposes _parse_network_breakpoint_input.
src/modelskill/obs.py Adds EdgeObservation, gtype="edge", and node/breakpoint alias docs.
src/modelskill/network.py Adds selective loading for Res1D, safer __repr__, and a copy() helper.
src/modelskill/model/network.py Extends NetworkModelResult to extract by node alias/breakpoint and by edge.
src/modelskill/model/adapters/_res1d.py Refactors Res1D adapters to support optional data loading for nodes/gridpoints.
src/modelskill/matching.py Allows matching EdgeObservation to extracted NodeModelResult.
src/modelskill/comparison/_comparison.py Tightens typing for raw_mod_data.
src/modelskill/comparison/_comparer_plotter.py Adds mypy return-value suppression for populated axes list.
src/modelskill/comparison/_collection_plotter.py Adds mypy return-value suppression for populated axes list.
src/modelskill/init.py Re-exports EdgeObservation.
docs/user-guide/network.qmd Documents selective loading, aliases, breakpoint usage, and edge observations.
docs/_quarto.yml Adds EdgeObservation to docs navigation and quartodoc sections.
docs/.gitignore Ignores generated *.quarto_ipynb notebooks.
Comments suppressed due to low confidence (1)

src/modelskill/model/network.py:203

  • _extract_node() resolves string/tuple aliases via self.network._alias_map but does not verify that the resolved node_id is actually present in self.data.node (which can happen with selective loading). In that case self.data.sel(node=...) will raise an unhelpful KeyError. Consider validating membership after _resolve_alias() and raising a ValueError explaining that the node exists in the topology but its timeseries was not loaded (and how to load it via Network.from_res1d(nodes=..., reaches=...)).
    def _extract_node(self, observation: NodeObservation) -> NodeModelResult:
        if observation.at is None and observation.node is None:
            raise ValueError("NodeObservation must have either 'node' or 'at' set")

        raw_id = observation.at if observation.at is not None else observation.node
        assert raw_id is not None  # Redundant assertion, included for mypy
        node_id = self._resolve_alias(raw_id)

        return NodeModelResult(
            data=self.data.sel(node=node_id).drop_vars("node"),
            node=node_id,
            name=self.name,

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/modelskill/model/network.py Outdated
Comment thread src/modelskill/obs.py Outdated
Comment thread src/modelskill/model/adapters/_res1d.py
Comment thread docs/user-guide/network.qmd Outdated
Comment thread docs/user-guide/network.qmd Outdated
Comment thread src/modelskill/model/network.py Outdated
Comment thread src/modelskill/model/network.py Outdated
jpalm3r and others added 2 commits April 22, 2026 15:08
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jpalm3r and others added 4 commits April 22, 2026 15:12
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automate finding by reach (without chainage) for relevant quantities

4 participants