Skip to content

Optimize I/O and add hourly LSTM support for summed Q' validation#187

Merged
taddyb merged 4 commits into
DeepGroundwater:masterfrom
taddyb:parameters
Apr 20, 2026
Merged

Optimize I/O and add hourly LSTM support for summed Q' validation#187
taddyb merged 4 commits into
DeepGroundwater:masterfrom
taddyb:parameters

Conversation

@taddyb
Copy link
Copy Markdown
Collaborator

@taddyb taddyb commented Apr 19, 2026

Summary

  • StreamflowReader: Replace xr.interp(method="nearest") with np.repeat(..., 24) for daily→hourly interpolation, eliminating xarray interpolation overhead on every training batch
  • summed_q_prime.py: Pre-filter divides to only upstream basins (mirrors training dataloader), use isel with integer indices instead of lazy sel/resample, GPU-accelerate per-gauge nansum via CuPy, and collapse hourly→daily via numpy reshape+mean on CPU before moving to GPU
  • CLAUDE.md: Comprehensive rewrite with full module map, config system, training/testing pipeline, temporal resolution handling, and dataset layer documentation
  • Tests: New test suite for summed Q' validating hourly/daily equivalence, pre-filtered divide correctness, store offset handling, and CuPy vs NumPy reference matching
  • clean_lstm_lateral_inflows.py: Added to references for LSTM data preparation

Test plan

  • uv run pytest tests/scripts/test_summed_q_prime.py — 7 tests pass
  • uv run pytest tests/io/test_readers.py — 28 tests pass
  • Pre-commit hooks (ruff, mypy, format) all pass
  • Daily LSTM summed Q' output verified identical (float32 tolerance) to v0.5.2 baseline across 2808 gauges

🤖 Generated with Claude Code

taddyb and others added 4 commits April 19, 2026 16:05
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@taddyb taddyb merged commit c9a0242 into DeepGroundwater:master Apr 20, 2026
4 checks passed
@taddyb taddyb deleted the parameters branch April 20, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant