Summary
At v1.x, the SDK has asymmetric handling of out-of-range date requests: NWP forecasts validate upfront and raise loudly, while every observation/climate source silently returns empty results. For a quant-research SDK whose value prop is leakage-free training pairs, silent empty results are exactly the failure mode that corrupts training sets without anyone noticing.
This issue proposes a production solution: a source-coverage manifest + upfront validation + one new exception class, mirroring the HistoricalDepthError pattern that already works well for NWP.
Current behavior
NWP forecasts — explicit, loud (good)
_nwp_cycle_chunks.py:89-228 defines a hardcoded NWP_HISTORICAL_DEPTH table (HRRR ≥ 2014-07-30, GFS ≥ 2021-01-01, live-only models capped at 7 days). check_historical_depth(model, cycle) runs before any HTTP call and raises HistoricalDepthError carrying model, requested_cycle, archive_depth.
Observations / climate — silent, graceful-degrade (problematic)
research() accepts from_date/to_date with zero upfront validation:
No `DataAvailabilityError`, no "requested range exceeds source coverage" warning. A user asking for `2010-01-01 → 2010-12-31` (before AWC backfill, before GHCNh for many stations) gets an empty DataFrame, not an error.
What's already in place (partial)
…but both only report what's in our local cache, not what the upstream source actually covers. That's the missing layer.
Why this matters at v1.x
- Leakage-sensitive use case. Quants training NHIGH/NLOW models can't distinguish "no data for this range" from "empty result because we asked wrong" — both surface as a zero-row DataFrame.
- Agent / MCP consumers (v0.2 roadmap) need a queryable surface to know what's available before constructing a query. Right now they'd have to probe by trial.
- The NWP fix already exists and works well. The pattern just needs extending to observations.
Proposed design
Layer 1 — Source-coverage manifest
New module: `packages/weather/src/mostlyright/weather/_sources/coverage.py`
```python
@DataClass(frozen=True)
class SourceCoverage:
source: str # "awc" | "iem_asos" | "ghcnh" | "nws_cli" | "iem_mos"
earliest: date | str | None # date, or "per_station" / "per_wfo" sentinel, or None=unbounded
latest_offset: timedelta | None # latest = now - offset; None = up to now
rolling_window: timedelta | None # for AWC's 168hr cap
station_fn: Callable[[str], tuple[date, date]] | None # per-station bound lookup
SOURCE_COVERAGE: dict[str, SourceCoverage] = {
"awc": SourceCoverage("awc", None, None, timedelta(hours=168), None),
"iem_asos": SourceCoverage("iem_asos", "per_station", timedelta(minutes=15), None, _iem_station_bounds),
"ghcnh": SourceCoverage("ghcnh", "per_station", timedelta(days=1), None, _ghcnh_station_bounds),
"nws_cli": SourceCoverage("nws_cli", "per_wfo", timedelta(hours=24), None, _wfo_bounds),
"iem_mos": SourceCoverage("iem_mos", date(2007,1,1), timedelta(hours=6), None, None),
}
```
Per-station bounds for IEM ASOS / GHCNh come from a manifest file shipped with the wheel (built from IEM's `network=ASOS` metadata API). For unknown stations, fall back to a permissive default + warn on first 404. Refreshable via a `mostlyright refresh-coverage` CLI command.
Layer 2 — One new exception class
`packages/core/src/mostlyright/core/exceptions.py`:
```python
class DataAvailabilityError(TradewindsError):
default_error_code = "DATA_AVAILABILITY"
# attributes:
# source: str
# station: str | None
# requested: tuple[date, date]
# available: tuple[date | None, date | None]
# kind: Literal[
# "no_overlap", # request entirely outside coverage
# "partial_before", # request starts before earliest
# "partial_after", # request ends after latest
# "rolling_window", # request exceeds rolling cap (AWC)
# "unknown_station", # station not in coverage manifest
# ]
```
Mirrors `HistoricalDepthError`'s shape exactly (already battle-tested in NWP).
Layer 3 — Upfront validation at public entry points
Every public entry point — `research()`, `observations()`, `climate()`, the forecast helpers — runs `validate_coverage()` before the first HTTP call. Behavior controlled by a single kwarg:
```python
research(..., on_unavailable: Literal["raise", "clamp", "warn"] = "warn")
```
| Mode |
Behavior |
Use case |
| `raise` |
`DataAvailabilityError` on any gap |
Production training pipelines, CI fixtures |
| `clamp` |
Silently clamp to available range, attach `df.attrs["coverage_note"]` |
Notebook exploration |
| `warn` (default) |
Clamp + `UserWarning` naming source + clamped range |
Best default — loud signal, soft failure |
Layer 4 — Public query surface
```python
mostlyright.coverage(source: str | None = None, station: str | None = None) -> DataAvailability
```
Lets agents/users discover what's available before constructing a query. Merges source-manifest data with cache state (extending existing `discovery.availability(station)`).
Key tradeoff: default `warn` vs default `raise`
- `raise`-by-default is the leakage-safe choice but forces every notebook user to know exact source bounds before any query — high friction for a tool whose value prop includes discoverability.
- `warn`-by-default gets the safety signal in logs / pytest output without breaking exploration. CI users opt into `raise`.
Recommendation: ship `warn` as default; document `raise` as the production-pipeline pattern. Open to flipping if user research says otherwise.
Migration plan (phased, non-breaking)
- Phase 1 — Manifest + exception + `validate_coverage()` helper. No call-site changes. Ships behind `mostlyright.coverage.check(source, station, range)` so users can adopt early.
- Phase 2 — Wire `on_unavailable` kwarg into `research()`, `observations()`, `climate()`, forecast helpers. Default `warn`.
- Phase 3 — Extend `discovery.availability(station)` to merge cache state with source coverage. Single call answers both "what can I get" and "what do I have".
- Phase 4 — Refreshable per-station manifest via packaged data file + CLI command (`mostlyright refresh-coverage`).
Each phase is independently shippable and additive (no breaking changes until a future major version flips the default to `raise`, if ever).
Open questions
- Where should the per-station manifest live? Shipped in `mostlyrightmd-weather` wheel (~MB scale) vs lazy-fetched from IEM on first use vs hybrid (small canonical set in wheel, full set lazy)?
- Should `coverage()` hit the network? If yes (live freshness probe), it's slow but accurate. If no (manifest-only), it's instant but can drift. Probably manifest-only with a `live=True` opt-in.
- NWP integration. `HistoricalDepthError` predates this design. Keep it as a subclass of `DataAvailabilityError`, or leave parallel? Subclassing gives users one `except` to catch all date-range issues.
- TS Parity. This is a public-API change — needs the Dual-SDK Planning Rule TS Parity section when this becomes a phase plan. Likely identical API, adapted implementation (no pandas in TS).
- Catalog "backfill jobs" for stations with non-obvious coverage (e.g., a station that has IEM data from 1995 but GHCNh data from 2007 — coverage is source-specific even within one station). Manifest must be `(source, station)` keyed, not just `station`.
Acceptance criteria
References
Summary
At v1.x, the SDK has asymmetric handling of out-of-range date requests: NWP forecasts validate upfront and raise loudly, while every observation/climate source silently returns empty results. For a quant-research SDK whose value prop is leakage-free training pairs, silent empty results are exactly the failure mode that corrupts training sets without anyone noticing.
This issue proposes a production solution: a source-coverage manifest + upfront validation + one new exception class, mirroring the
HistoricalDepthErrorpattern that already works well for NWP.Current behavior
NWP forecasts — explicit, loud (good)
_nwp_cycle_chunks.py:89-228defines a hardcodedNWP_HISTORICAL_DEPTHtable (HRRR ≥ 2014-07-30, GFS ≥ 2021-01-01, live-only models capped at 7 days).check_historical_depth(model, cycle)runs before any HTTP call and raisesHistoricalDepthErrorcarryingmodel,requested_cycle,archive_depth.Observations / climate — silent, graceful-degrade (problematic)
research()acceptsfrom_date/to_datewith zero upfront validation:No `DataAvailabilityError`, no "requested range exceeds source coverage" warning. A user asking for `2010-01-01 → 2010-12-31` (before AWC backfill, before GHCNh for many stations) gets an empty DataFrame, not an error.
What's already in place (partial)
DataAvailability/RangeInfomodels existdiscovery.availability(station)exists…but both only report what's in our local cache, not what the upstream source actually covers. That's the missing layer.
Why this matters at v1.x
Proposed design
Layer 1 — Source-coverage manifest
New module: `packages/weather/src/mostlyright/weather/_sources/coverage.py`
```python
@DataClass(frozen=True)
class SourceCoverage:
source: str # "awc" | "iem_asos" | "ghcnh" | "nws_cli" | "iem_mos"
earliest: date | str | None # date, or "per_station" / "per_wfo" sentinel, or None=unbounded
latest_offset: timedelta | None # latest = now - offset; None = up to now
rolling_window: timedelta | None # for AWC's 168hr cap
station_fn: Callable[[str], tuple[date, date]] | None # per-station bound lookup
SOURCE_COVERAGE: dict[str, SourceCoverage] = {
"awc": SourceCoverage("awc", None, None, timedelta(hours=168), None),
"iem_asos": SourceCoverage("iem_asos", "per_station", timedelta(minutes=15), None, _iem_station_bounds),
"ghcnh": SourceCoverage("ghcnh", "per_station", timedelta(days=1), None, _ghcnh_station_bounds),
"nws_cli": SourceCoverage("nws_cli", "per_wfo", timedelta(hours=24), None, _wfo_bounds),
"iem_mos": SourceCoverage("iem_mos", date(2007,1,1), timedelta(hours=6), None, None),
}
```
Per-station bounds for IEM ASOS / GHCNh come from a manifest file shipped with the wheel (built from IEM's `network=ASOS` metadata API). For unknown stations, fall back to a permissive default + warn on first 404. Refreshable via a `mostlyright refresh-coverage` CLI command.
Layer 2 — One new exception class
`packages/core/src/mostlyright/core/exceptions.py`:
```python
class DataAvailabilityError(TradewindsError):
default_error_code = "DATA_AVAILABILITY"
# attributes:
# source: str
# station: str | None
# requested: tuple[date, date]
# available: tuple[date | None, date | None]
# kind: Literal[
# "no_overlap", # request entirely outside coverage
# "partial_before", # request starts before earliest
# "partial_after", # request ends after latest
# "rolling_window", # request exceeds rolling cap (AWC)
# "unknown_station", # station not in coverage manifest
# ]
```
Mirrors `HistoricalDepthError`'s shape exactly (already battle-tested in NWP).
Layer 3 — Upfront validation at public entry points
Every public entry point — `research()`, `observations()`, `climate()`, the forecast helpers — runs `validate_coverage()` before the first HTTP call. Behavior controlled by a single kwarg:
```python
research(..., on_unavailable: Literal["raise", "clamp", "warn"] = "warn")
```
Layer 4 — Public query surface
```python
mostlyright.coverage(source: str | None = None, station: str | None = None) -> DataAvailability
```
Lets agents/users discover what's available before constructing a query. Merges source-manifest data with cache state (extending existing `discovery.availability(station)`).
Key tradeoff: default `warn` vs default `raise`
Recommendation: ship `warn` as default; document `raise` as the production-pipeline pattern. Open to flipping if user research says otherwise.
Migration plan (phased, non-breaking)
Each phase is independently shippable and additive (no breaking changes until a future major version flips the default to `raise`, if ever).
Open questions
Acceptance criteria
References