Fixes for data types and other common sources of errors

[jni]

Description

I recently mentioned on the mailing list that a large proportion of our errors come from new (and old!) users surprised by the behavior of our data type conversions.

Although they are documented, this documentation is not prominent enough. Indeed, we barely even cover these in our scikit-image tutorials! Additionally, our own code base really is not careful enough to guide users along.

I want this issue to compile both common errors due to data types and proposed solutions, including a to-do list of consensus solutions.

Common errors

(Please add links here if you find them.)

• User has a 12-bit image that is encoded in a uint16 array. When this is converted to uint8 or float, the result is an image that has much lower contrast (16x) than expected.
• User has a float image with values falling outside the range [0, 1] or [-1, 1]. Some scikit-image functions fail altogether with this kind of input, while others warn. In almost all cases, these warnings are unnecessary.
• User wants to apply scikit-image functions to an integer array. This array, by default, is of type int32 or int64. scikit-image converts the values in the array to float by dividing by 2^31, resulting in tiny values (which for example are impossible to encode in common image formats)

Proposed solutions

• Figure out image metadata format and handling. Having correct metadata (bit_depth=12, for example) would alleviate many issues.
• Clearer separation of data viewing and data processing. See email by @ds604. Specifically, we could tweak io.imshow to clearly display characteristics of the input data, including text info in the figure title/subtitle, colorbar, dtype, etc. Perhaps revisiting skimage.viewer as an integral part of skimage is warranted.

Rejected proposals

• Magically infer the range of input images (objections here and here.)

To-do list

• Add an FAQ section to the website homepage.
• Audit the code base to make sure that float images work unimpeded wherever it's possible. (Search for img_as_float calls.) (Detailed suggestion by @grlee77 here.)
• Remove automatic range conversion for int32 and int64 images.
• Warn on conversion of uint16 images where no value exceeds 4096 (possible 12-bit image)
• Add bit-depth option to in_range and out_range parameters for exposure.rescale_intensity.

[soupault]

xref #2605

[stefanv]

Remove automatic range conversion for int32 and int64 images.

How would we do this, given that we typically convert to floating point images before processing?

[stefanv]

For metadata, perhaps switching to a single I/O backend per format would help.

[jni]

@stefanv

Remove automatic range conversion for int32 and int64 images.
How would we do this, given that we typically convert to floating point images before processing?

Something along the lines of preserve_range=True implicitly if the dtype is int higher than uint16. And before you complain about the magic of this, we are already magically treating uint32 images differently from uint16 ones!

[hmaarrfk]

Personally, I'm against constantly checking bounds. If I want to perform convolutions and scaling on my images, that should be my choice. most algorithms are scale invariant, or require some kind of normalization anyway.

the speed of the algorithms shouldn't be impeded by range checks on my giant matricies multiple times per calls.

what is the point of imshow really? what is deficient in matplotlib.pyplot.imshow? performance, yes, but it autoscales and allows you to do things like add colorbars when you want to. Using it also empowers the user to learn a useful other tool so that they can expand on their skillset later.

I think visualization should be left to the user and not to the processing library.

[hmaarrfk]

this kinda relates to my PR #3062 where I dont think we should impose 64 bit floats on users.

I would suggest that if you want to make it new user proof, that you change the image load functions. Those happen once at the beginning and probably have had little processing applied to them.

Many of the functions first call img_as_float. loading images should arguably just convert it to the float by default, alleviating many issues with subsequent conversions.

[emmanuelle]

Could we consider dropping range conversion altogether (like for 1.0 version)?

[jni]

To answer that, we need to figure out all the places where the assumed range is useful. There is at least one case: RGB float images being displayed by matplotlib are clipped to [0, 1]:

In [1]: import numpy as np

In [2]: import matplotlib.pyplot as plt

In [3]: image = np.random.random((10, 10, 3)) * 255

In [4]: plt.imshow(image)
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Out[4]: <matplotlib.image.AxesImage at 0x7f204a2e0a90>

I don't want to coordinate a deprecation with matplotlib.

It also matters when downcasting: what do you do when you have out-of-range values? I don't want to trade rescaling error reports for overflow error reports.

[emmanuelle]

OK thanks. Do we do a lot of downcasting?

[jni]

float -> uint8 is very very very common. I presume that anyone that does uint16 (e.g. 12-bit image) -> float will end up saving their image as uint8.