Summary
The A* pathfinding implementation pre-allocates dense numpy arrays at the full grid dimensions, even when given dask-backed input. For large rasters this makes pathfinding completely unusable.
OOM Code Paths
| Line |
Code |
Memory (30 TB raster) |
pathfinding.py:~285-312 |
dist = np.full((height, width), np.inf) |
30 TB |
pathfinding.py:~285-312 |
visited = np.zeros((height, width)) |
30 TB |
For a 30 TB raster (~3.75 trillion pixels at float64), these arrays alone require ~60 TB.
Severity
CRITICAL — guaranteed OOM with large dask-backed input.
Suggested Fix
- For short paths: use a bounded A* search with a maximum search radius, only allocating arrays for the search region
- For long paths: implement hierarchical pathfinding (HPA*) that works at multiple resolution levels
- At minimum: raise a clear error when the grid exceeds available memory rather than silently OOMing
Summary
The A* pathfinding implementation pre-allocates dense numpy arrays at the full grid dimensions, even when given dask-backed input. For large rasters this makes pathfinding completely unusable.
OOM Code Paths
pathfinding.py:~285-312dist = np.full((height, width), np.inf)pathfinding.py:~285-312visited = np.zeros((height, width))For a 30 TB raster (~3.75 trillion pixels at float64), these arrays alone require ~60 TB.
Severity
CRITICAL — guaranteed OOM with large dask-backed input.
Suggested Fix