Selective/lazy from_res1d ingestion by quantity, for calibration loops

[jpalm3r]

Summary

from_res1d takes about 18-25 seconds on a moderately sized network file. The documented nodes=[...], reaches=[] filtering cuts that to around 5 seconds, but the remaining time is still mostly spent building topology rather than reading data.

In a calibration loop that rebuilds the network for every trial, this dominates the runtime — matching itself is already fast in comparison.

Request

Support selective or lazy ingestion by quantity, on top of the existing node/reach filtering, so calibration workflows can build only what a given trial needs.

Relevant entry points

• Network.from_res1d (src/modelskill/network.py:351-447)
• Network._load_res1d_network (network.py:448-487)

Found by a test user evaluating the network functionality ahead of a 1.4.0 release.

[jpalm3r]

I looked into this. Summary of what I found and a proposed approach.

How loading works today

from_res1d opens the file with no filters, then iterates every reach to build the full
topology. Per-location time series are read only for the selected nodes/reaches, via
node.to_dataframe(), which reads all quantities at that location.

Two separate costs

1. Topology — the loop over all reaches touches each reach's metadata and every
   gridpoint's chainage over the pythonnet/.NET boundary (376 gridpoints in the test file).
2. Data reads — the to_dataframe() calls, already gated by nodes/reaches.

On the small test file topology is ~40% of load time; on a large file it dominates, which
matches your report that the ~5s left after nodes=[...], reaches=[] is mostly topology.
A quantity filter reduces the data-read cost, not the topology cost. Worth flagging up
front so expectations are right.

mikeio1d already supports quantities=, but with a catch

Res1D(path, quantities=[...]) is fast, but it drops every location that lacks that
quantity — e.g. a WaterLevel-only node raises KeyError under a ["Discharge"] filter.
Passing the filter to the constructor would break the current guarantee that the full
topology is always built and that find()/recall() work on every node.

Per-location quantity reads do exist (node.WaterLevel.to_dataframe()), so we can filter
which quantities are read while still enumerating the full topology from an unfiltered file.

Proposed approach

Add a quantities parameter to from_res1d mirroring the nodes/reaches semantics
(None = all, str/list = subset, [] = none), applied at the read layer so the full
topology is preserved. A location that does not carry a requested quantity becomes
topology-only rather than an error.

Caveat and follow-up

This gives selective ingestion by quantity as requested, but does not touch the topology
building that dominates on large files. If calibration-loop speed is the real target, the
bigger win is likely reusing the topology across trials (build once, re-read only data per
trial) — worth a separate issue.