Skip to content

Speed up focal and proximity test suites #1931

@brendancol

Description

@brendancol

Reason or Problem

The focal and proximity test suites take about 58 seconds locally. A few specific patterns dominate the runtime:

  • test_apply_rejects_oversize_kernel_1284, test_focal_stats_rejects_oversize_kernel_1284, and test_hotspots_rejects_oversize_kernel_1284 each allocate a (50001, 50001) float32 kernel just to trip the memory-guard MemoryError. The np.ones call alone takes about 5 seconds and pins 10 GB of RAM, even though the guard only does shape arithmetic against _available_memory_bytes().
  • test_focal.py builds data_random_sparse and data_gaussian at import time. Neither is referenced anywhere in the file.
  • test_mean_transfer_function_dask_cpu and test_mean_transfer_function_dask_gpu chunk a 100x100 array with chunks=(3, 3), producing a 1156-chunk dask graph. The dask path runs the same with a handful of chunks.
  • Several pytest.mark.parametrize sweeps over (boundary, size, chunks) repeat the same cross-backend assertion across redundant combinations.

Proposal

Reduce the test wall-clock without losing coverage:

  • Patch xrspatial.focal._available_memory_bytes to return a tiny value and use a small kernel in the three oversize-kernel tests. The guard still fires and the test still asserts the same behavior, but we skip the 10 GB allocation.
  • Delete the unused module-level data_random_sparse and data_gaussian.
  • Use larger dask chunks in the mean-transfer-function dask tests.
  • Trim the (size, chunks) matrix in the boundary parity tests to keep small, medium, and chunk-equals-array cases without duplicating mid-range combinations.

Value

Faster feedback for anyone running or modifying tests in xrspatial.focal or xrspatial.proximity, and shorter CI for PRs that touch either module.

Drawbacks

Lower-priority shape combinations get dropped from the parametrize matrix. If a regression hides specifically in one of those combinations, it could slip through.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfocal toolsFocal statistics and hotspot analysisinfrastructureCI, benchmarks, and toolingproximity toolsProximity, allocation, direction, cost distance

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions