Summary
sink_d8() (xrspatial/hydro/sink_d8.py) runs the BFS connected-component kernel on the eager numpy and cupy backends with no memory guard. A large in-memory raster can OOM the process before any sanity check runs.
Same pattern as the recent hydro guard series (#1318/#1319, #1322/#1325, #1327/#1332, #1334/#1336, #1337/#1341, #1338/#1340, #1339/#1342, #1343/#1347, #1344/#1346, #1345/#1348, #1349/#1352, #1350/#1354, #1351/#1353).
Numpy path: ~24 bytes per pixel of working set
_sink_cpu allocates three full-size buffers beyond the input:
labels float64: 8 B/pxqueue_r int64: 8 B/pxqueue_c int64: 8 B/px
Total ~24 B/px. A 30000x30000 input needs ~21 GB of working memory on top of the 7 GB float64 cast of flow_dir itself.
Cupy path: ~8 bytes per pixel of working set
_sink_cupy allocates labels (float64, 8 B/px) plus a constant-size changed flag. The flow_dir_f64 cast is counted against the input.
Worked example
A 25000x25000 D8 grid:
- numpy working set: 25000 * 25000 * 24 = 15.0 GB
- cupy working set: 25000 * 25000 * 8 = 5.0 GB
A 16 GB workstation OOMs on the numpy path. An 8 GB consumer GPU OOMs on the cupy path.
Proposed fix
Same shape as #1319: per-module _BYTES_PER_PIXEL (24) and _GPU_BYTES_PER_PIXEL (8), _check_memory / _check_gpu_memory helpers, 50% threshold, dispatch wires the guards into the eager branches only. Dask backends skip the guard since per-tile allocations are bounded.
Summary
sink_d8()(xrspatial/hydro/sink_d8.py) runs the BFS connected-component kernel on the eager numpy and cupy backends with no memory guard. A large in-memory raster can OOM the process before any sanity check runs.Same pattern as the recent hydro guard series (#1318/#1319, #1322/#1325, #1327/#1332, #1334/#1336, #1337/#1341, #1338/#1340, #1339/#1342, #1343/#1347, #1344/#1346, #1345/#1348, #1349/#1352, #1350/#1354, #1351/#1353).
Numpy path: ~24 bytes per pixel of working set
_sink_cpuallocates three full-size buffers beyond the input:labelsfloat64: 8 B/pxqueue_rint64: 8 B/pxqueue_cint64: 8 B/pxTotal ~24 B/px. A 30000x30000 input needs ~21 GB of working memory on top of the 7 GB float64 cast of
flow_diritself.Cupy path: ~8 bytes per pixel of working set
_sink_cupyallocateslabels(float64, 8 B/px) plus a constant-sizechangedflag. Theflow_dir_f64cast is counted against the input.Worked example
A 25000x25000 D8 grid:
A 16 GB workstation OOMs on the numpy path. An 8 GB consumer GPU OOMs on the cupy path.
Proposed fix
Same shape as #1319: per-module
_BYTES_PER_PIXEL(24) and_GPU_BYTES_PER_PIXEL(8),_check_memory/_check_gpu_memoryhelpers, 50% threshold, dispatch wires the guards into the eager branches only. Dask backends skip the guard since per-tile allocations are bounded.