Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

imshow doesn't normalize the color range in RGB images #9391

Closed
michalkahle opened this issue Oct 13, 2017 · 17 comments

Comments

Projects
None yet
7 participants
@michalkahle
Copy link

commented Oct 13, 2017

imshow doesn't normalize the color range in RGB images

Bug summary
When single channel float image is passed to imshow it gets normalized to range [0,1] before display. This does not happen for RGB images.

Code for reproduction

img = np.arange(0, 0.6, .1).reshape(6,1)
plt.subplot(141)
plt.title('grayscale')
plt.imshow(img)
plt.colorbar()

img = np.arange(0, 0.6, .1).repeat(3).reshape(6,1,3)
plt.subplot(142)
plt.title('RGB')
plt.imshow(img)
plt.colorbar()

img = np.arange(0, 1.6, .2).reshape(8,1)
plt.subplot(143)
plt.title('grayscale')
plt.imshow(img)
plt.colorbar()

img = np.arange(0, 1.6, .2).repeat(3).reshape(8,1,3)
plt.subplot(144)
plt.title('RGB')
plt.imshow(img)
plt.colorbar()

Actual outcome
mpl_color_norm
Please also note how the colorbars are misleading.

Expected outcome
I'd expect the RGB images to look the same as grayscale.

Real life example
This is how I encountered this issue. What is the value in all three channels at [3,18]? It is actually around 1.2!
download

Matplotlib version: 2.0.2

Discussion
The same issue: #5382 , on SO, and another SO.

In #5382 @tacaswell explains that and RGB image "goes through a different code path which do not get passed through the normalize/colormap framework". Colormapping of course makes no sense for color images but I believe that normalization should be consistently applied to RGB images.

I got used to matplotlib normalization of single channel float images and the need of using vmin and vmax to avoid this normalization and intuitively expected RGB images to be treated consistently. As I noticed subsequently the docstrigs say that "the value for each component of MxNx3 and MxNx4 float arrays should be in the range 0.0 to 1.0." But I believe that to simply overflow without raising error or warning is not correct.

@jklymak

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

RGB means RGB. I don't think it should be normalized.

I think a case could be made that it shouldn't do a modulo past 1.0 or below 0.

@anntzer

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

I think it's either error out ("Errors should never pass silently."), or clipping (because sometimes you may end up with values just a bit below 0 or a bit above 1 due to fp imprecision, so "practicality beats purity.").

See also #8854.

@WeatherGod

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

@anntzer

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

that is definitely an issue too... :)

@tacaswell tacaswell added this to the 2.2 (next feature release) milestone Oct 13, 2017

@tacaswell

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

I am confused why it wraps...

Agree the colorbar should refuse to let it's self be created for an RGB(A) image.

@anntzer

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

It wraps because ultimately everything is cast to uint8 (essentially (uint8_t)(val * 0xff)).

I also don't think it's release critical, the wraparound is also present in 1.5.0 and probably since ever.

Upon further thought I think I prefer erroring on such inputs, possibly with a message that points to np.clip.

@michalkahle

This comment has been minimized.

Copy link
Author

commented Oct 16, 2017

It might have been unfortunate to label my data "RGB image". It would be better to call it three channel float images. uint8 RGB images should NOT be normalized.
I think the viewer should do the most useful thing by default. As a user I often just want to quickly see what is in my data without much fuss. With arrays of floats my data can take the whole range. The clipping or modulo destroy my data. If I get an error about wrong range I'd be forced to normalize the data myself in which case I'd wonder why the viewer does not do this for me as it does with single channel image. If I wanted to be specific I'd use the norm, vmin and vmax parameters just as I do with single channel image. Simple. Consistent.

@jklymak

This comment has been minimized.

Copy link
Contributor

commented Oct 16, 2017

What three-channel data do you look at that isn’t meant to be an RGB image and why do you look at it as an image?

@michalkahle

This comment has been minimized.

Copy link
Author

commented Oct 17, 2017

@jklymak these are three repeated absorbance measurements in 384 well plate. I just wanted to quickly see the pattern on the plate as well as variability in the repeated measurements. I agree this is not typical use of 3 channel images.

@jklymak

This comment has been minimized.

Copy link
Contributor

commented Oct 17, 2017

Cool. OTOH I’m sure you can appreciate that it is wrong to automatically normalize an arbitrary RGB image. If the whole image is dark, it should stay dark unless the user wants to explicitly lighten it.

@michalkahle

This comment has been minimized.

Copy link
Author

commented Oct 18, 2017

I absolutely agree with that in case of integer images. But in case of say 16 bit per channel RGB images we want to show the whole range (not clip or wrap around). In case of float images we cannot show the whole range (the image would be almost always gray). Forcing the user to normalize the image to a range [0, 1] is a solution but I feel too restrictive one. Float images would be mostly used for image processing. While these are mostly single channel images, color images can be used too with tools like adapt_rgb. Please also note in the adapt_rgb examples they normalize rescale_intensity(1 - sobel_each(image)). I wondered why the 1 was needed. If you omit it the image gets normalized to [-1, 1] and the colors get wrapped around by imshow.

@WeatherGod

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

@jklymak

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2017

imshow accepts floats between 0. and 1 or uint8. I agree that it shouldn't round floats to uint8 unless it has to for rendering. I agree it probably shouldn't wrap, and clipping between 0 and 1 makes sense to me, or returning an error. I think it should definitely not normalize arbitrary data between 0 and 1.

It'd probably be easy to add support for 16-bit integers given that a lot of images are now 16 bits. But I guess there is a bit of an issue telling between a dark 16-bit image and an 8-bit image if the user doesn't specifically cast to the right-size uint.

@Zac-HD Zac-HD referenced this issue Jan 8, 2018

Merged

Support RGB[A] arrays in plot.imshow() #1796

4 of 4 tasks complete
@Zac-HD

This comment has been minimized.

Copy link
Contributor

commented Jan 10, 2018

tacaswell, I've recently been working on imshow in Xarray, where we concluded that the proper place for scaling and clipping fixes was upstream (ie here). My plan:

  • Support norm or vmin and vmax arguments with RGB[A] images. Among other things, this makes it quite trivial to support high-bit-depth images - just pass vmin=0, vmax=2**12 for 12-bit images, for example. Rejected by consensus on pull request.
  • If data are outside the valid range after scaling - that is, [0 .. 255] for integer types and [0 .. 1] for floating types - log a warning and then clip to the valid range before casting, to clip instead of modulo. Without this any pixel could represent a much higher or lower value than it seems.
  • Open a new issue for "colorbar should refuse to let it's self be created for an RGB(A) image" - it's a logically distinct change and I don't have the time to do both right now.

What do people think? If this seems reasonable I'll write a draft tomorrow 😄

@Zac-HD Zac-HD referenced this issue Jan 10, 2018

Merged

Clip RGB data to valid range for imshow #10220

5 of 5 tasks complete
@anntzer

This comment has been minimized.

Copy link
Contributor

commented Jan 10, 2018

Untagged from release critical as neither the wrapping and the inappropriate colorbar'ing are "new" bugs, and RGB normalization is another can of worms (see discussion followup in #10220).

@michalkahle

This comment has been minimized.

Copy link
Author

commented Jan 11, 2018

@Zac-HD I like the proposal. Even without normalization the out-of-range warning and clipping instead of modulo would still be nice improvement.

@Zac-HD

This comment has been minimized.

Copy link
Contributor

commented Jan 11, 2018

That's what I'm doing in the pull request!

(and normalisation will happen downstream in Xarray, which will cover most of my use at least 😉 )

@QuLogic QuLogic modified the milestones: needs sorting, v2.2.0 Feb 12, 2018

@matplotlib matplotlib deleted a comment from biswajitcsecu Sep 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.