Skip to content

Geotiff polish: validation, caching caps, parallelism thresholds, memory guards (#1488)#1493

Merged
brendancol merged 2 commits intomainfrom
issue-1488
May 5, 2026
Merged

Geotiff polish: validation, caching caps, parallelism thresholds, memory guards (#1488)#1493
brendancol merged 2 commits intomainfrom
issue-1488

Conversation

@brendancol
Copy link
Copy Markdown
Contributor

Fixes #1488.

What's in here

Ten small ergonomic / safety items from the geotiff audit, bundled.

  • C-1: to_geotiff validates compression up-front instead of letting the error surface from _compression_tag. Same exception class, friendlier traceback.
  • C-2: Documented the read_geotiff_dask -> read_vrt defensive fallback. No public behavior change.
  • C-5: write_vrt docstring spells out relative, crs_wkt, nodata instead of pointing at the internal helper.
  • C-6: Predictor doc lists True/2 (horizontal), False/0/1 (none), 3 (fp, float dtypes).
  • C-7: tiled=False with a non-default tile_size now warns instead of silently ignoring.
  • P-3: _mmap_cache got a soft LRU cap (default 32, configurable via XRSPATIAL_GEOTIFF_MMAP_CACHE_SIZE). In-use entries are never evicted; only idle ones drop when the cap is exceeded.
  • P-4: Decode parallelism now triggers on n_tiles > 1 and tile_pixels > 64*1024 regardless of compression. Uncompressed multi-tile reads previously skipped the pool.
  • P-5: read_geotiff_dask now raises ValueError (with a hint) when chunks would produce more than 50,000 dask tasks, instead of silently rescaling at 1M. The hint suggests a chunk size that fits.
  • P-6: _gpu_decode._check_gpu_memory(required_bytes) calls cupy.cuda.runtime.memGetInfo() and raises MemoryError when the request exceeds 90% of free VRAM. Wired into the four largest allocations (nvJPEG output buffer, full-image output buffer, tile decode buffer x2).
  • P-9: COG auto-overview generation capped at 8 levels (1/256 resolution). Explicit overview_levels=[...] is not capped.

Behavior changes

  • to_geotiff(..., compression=<bad>) now raises with "Unknown compression" rather than "Unsupported compression". Existing test updated to match either phrasing.
  • to_geotiff(..., tiled=False, tile_size=<non-default>) now emits a UserWarning.
  • read_geotiff_dask raises instead of auto-scaling past 50K tasks. Callers relying on the old silent rescale will need to pick chunks explicitly.

Test plan

  • 24 new tests in test_polish_1488.py, one cluster per item
  • All 493 pre-existing geotiff tests still pass (4 unrelated matplotlib failures from the env, not from this change)
  • Touched test (test_unsupported_compression) updated to accept either error phrasing

@github-actions github-actions Bot added the performance PR touches performance-sensitive code label May 5, 2026
@brendancol brendancol merged commit 6cd69f1 into main May 5, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance PR touches performance-sensitive code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Geotiff polish: validation, caching caps, parallelism thresholds, memory guards

1 participant