SLIC segmentation results differ depending on input image datatype

[loicdtx]

I just realized that segmentation output can vary a lot depending on the datatype of the input array.

See reproducible example below

from skimage import segmentation, data
import numpy as np
import matplotlib.pyplot as plt


img = data.cat()
labels_0 = segmentation.slic(img)
labels_1 = segmentation.slic(img.astype(np.float32))
fig, [ax0, ax1] = plt.subplots(1,2)
ax0.imshow(segmentation.mark_boundaries(img, labels_0))
ax1.imshow(segmentation.mark_boundaries(img, labels_1))
plt.show()

Version information

3.8.10 (default, Jun  2 2021, 10:49:15) 
[GCC 10.3.0]
Linux-5.8.0-63-generic-x86_64-with-glibc2.32
scikit-image version: 0.18.2
numpy version: 1.21.1

[grlee77]

I think it is not so much the datatype, but the scaling that matters here. Internally skimage.img_as_float(img) results in rescaling the uint8 image to floats in the range [0.0, 1.0]. The .astype call here converts to floats with values in the original integer range of img. This difference in absolute scale is the source of the difference.

Using the following gives labels_1 that looks the same as labels_0

img_f32 = img.astype(np.float32)
img_f32 /= img_f32.max()
labels_1 = segmentation.slic(img_f32)

For this algorithm I think we should always just rescale internally to [0.0, 1.0] range so the default compactness value is a reasonable starting point. A few releases back img_as_float always rescaled to [0, 1.0] but in more recent releases it preserves the original floating point range. We do in most cases want to move toward preserving range, but this function only outputs labels anyway, so I think it is best to just the input to [0.0, 1.0] so that the default settings are reasonable regardless of the input data scale.