fix(waterdata): Raise on mid-pagination failures instead of silently truncating

[thodson-usgs]

The paginated request loops in waterdata (_walk_pages and get_stats_data) have, since #273, logged page failures correctly but kept a "best effort" contract: when any follow-up page failed, log the failure and return whatever pages had been collected so far. That contract was the wrong default — the waterdata API exposes no resume cursor once a paginated walk is interrupted, so the partial DataFrame couldn't reliably be extended. Callers silently received truncated data they had no way to detect.

Both loops now raise on any mid-pagination failure (429, 5xx, network error). The wrapper RuntimeError carries:

• the number of pages successfully collected,
• the upstream cause (as both the message text and __cause__ for programmatic inspection),
• a short menu of recovery actions: wait and retry, reduce request size (fewer locations, shorter time range, smaller limit), or obtain an API token.

A small shared helper _paginated_failure_message builds the user-facing string so both loops stay aligned.

RuntimeError: Paginated request failed after collecting 7 page(s):
429: Too many requests made. Please obtain an API token or try again
later. To recover: wait for the rate-limit window to reset and retry,
reduce the request size (e.g. fewer locations, a shorter time range,
or a smaller ``limit``), or obtain an API token.

Behavior change

Callers that previously consumed partial DataFrames on transient upstream blips (5xx, network errors) will now see an exception they need to handle — typically: retry, possibly with a smaller limit or narrower query. Called out in NEWS.

This is intentional. With no resume API, "best effort partial" was just "silent truncation with friendly framing." A loud error is strictly safer; programmatic callers can try / except RuntimeError and use e.__cause__ (isinstance(..., requests.ConnectionError), status-code parsing, etc.) to branch on the failure mode.

Tests

Three new tests cover the new contract in both _walk_pages and get_stats_data — _raises_on_connection_error_mid_pagination, _raises_on_5xx_mid_pagination, _raises_on_mid_pagination_429. The two pre-existing best-effort tests were replaced (same scenarios, new assertion shape).

Ordering / relationship to other open PRs

This PR is independent of #276 (the multi-value GET chunker) and can land in either order.

This PR should land before #278 (the draft parallel chunker). Parallel sub-requests make the silent-truncation behavior easy to trigger empirically (one of two runs returned 330,174 rows instead of 337,808 with no exception). With this fix in place, the chunker can be enhanced separately to catch the RuntimeError and wrap it in QuotaExhausted with partial_frame — but the silent-data-loss bug is closed at the source.

Refs #273

[Copilot]

Pull request overview

This PR changes waterdata pagination behavior to fail loudly (raise) on any mid-pagination error rather than returning a silently truncated DataFrame, and updates tests + release notes to reflect the new contract.

Changes:

• Introduce _paginated_failure_message(...) and wrap mid-pagination exceptions in a RuntimeError that chains the original failure via __cause__.
• Update _walk_pages and get_stats_data to raise on mid-pagination failures (429/5xx/network errors) instead of returning partial results.
• Replace the prior “best-effort” pagination tests with new assertions that validate the raised exception and chaining behavior; document the behavior change in NEWS.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
dataretrieval/waterdata/utils.py	Adds shared failure-message helper and switches mid-pagination handling to raise with chained cause.
tests/waterdata_utils_test.py	Updates pagination-failure tests to assert raising behavior instead of logging/partial returns.
NEWS.md	Documents the behavior change for paginated waterdata requests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.