Skip to content

convolve_2d truncates float64 input to float32 #1096

@brendancol

Description

@brendancol

Describe the bug

All four convolve_2d backends hardcode .astype(np.float32) on input data and np.float32/f4 on output arrays. When float64 data is passed, half the mantissa bits are silently discarded.

Affected functions:

  • _convolve_2d_numpy (line 290): data = data.astype(np.float32)
  • _convolve_2d_dask_numpy (line 332): data = data.astype(np.float32)
  • _convolve_2d_cupy (line 396): data = data.astype(cupy.float32)
  • _convolve_2d_dask_cupy (line 405): data = data.astype(cupy.float32)

convolve_2d is used internally by convolution_2d, focal.mean, focal.hotspots, and other downstream functions.

Expected behavior

Integer inputs should be promoted to float32 (to avoid integer overflow in weighted sums). Float64 inputs should stay float64. Output dtype should match the promoted input dtype.

Reproduction

import numpy as np
from xrspatial.convolution import convolve_2d

data = np.array([[1e7 + 0.01, 1e7 + 0.02],
                 [1e7 + 0.03, 1e7 + 0.04],
                 [1e7 + 0.05, 1e7 + 0.06],
                 [1e7 + 0.07, 1e7 + 0.08]], dtype=np.float64)
kernel = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]], dtype=np.float64)

result = convolve_2d(data, kernel)
print(result.dtype)   # float32 -- should be float64
# The 0.0x differences are lost in float32

Affected code

xrspatial/convolution.py -- all four backend functions for convolve_2d

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingfocal toolsFocal statistics and hotspot analysis

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions