Skip to content

Add memory guard to viewshed numpy path #1229

@brendancol

Description

@brendancol

Describe the bug

_viewshed_cpu in xrspatial/viewshed.py allocates several arrays sized from the raster dimensions with no memory check. Peak working memory is around 500 bytes per pixel once you count the sort buffer, so a 20000x20000 raster (400M pixels) tries to allocate ~200 GB and either OOM-kills or thrashes swap.

The dask path already has a guard — see _viewshed_dask around line 2172: it checks r2_bytes against _available_memory_bytes() and raises MemoryError with a hint to use max_distance. The numpy path has nothing.

To Reproduce

import numpy as np
import xarray as xr
from xrspatial import viewshed

# Tries to allocate a multi-hundred-GB event_list before any check
arr = np.zeros((20000, 20000), dtype=np.float64)
xs = np.arange(20000, dtype=float)
ys = np.arange(20000, dtype=float)
r = xr.DataArray(arr, coords=dict(x=xs, y=ys), dims=["y", "x"])
viewshed(r, x=10000.0, y=10000.0, observer_elev=5)

Expected behavior

MemoryError before allocation, with a message pointing users at max_distance=.

Proposed fix

Guard at the top of _viewshed_cpu: estimate peak working memory (~500 * H * W bytes) via _available_memory_bytes() and raise if it would blow past a safe fraction of RAM.

Allocations that add up

  • event_list: 3 * H * W * 7 * 8 = 168 bytes/pixel
  • status_values + status_struct + idle: ~104 bytes/pixel
  • visibility_grid: 8 bytes/pixel
  • lexsort temporary: ~2x event_list during the sort

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinginput-validationInput validation and error messagesoomOut-of-memory risk with large datasetsviewshed tools

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions