Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not premultiply alpha when resizing with Image.NEAREST resampling #5304

Merged
merged 2 commits into from Mar 28, 2021

Conversation

@nulano
Copy link
Contributor

@nulano nulano commented Mar 3, 2021

Fixes #5300.

Test is based on the previous tests that make sure premultiplied alpha is used with BILINEAR resampling.

From https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.NEAREST:

Pick one nearest pixel from the input image. Ignore all other input pixels.

I would interpret that as meaning that each pixel in the output will be equal to one of the original pixels, i.e. not introducing new values.

Tests/test_image_transform.py Outdated Show resolved Hide resolved
@radarhere
Copy link
Member

@radarhere radarhere commented Mar 27, 2021

So, in what I presume is an extreme situation to allow for ease of testing, your test demonstrates that converting the image to pre-multiplied alpha and then back to normal alpha causes a loss of information when the alpha is zero - because any color with zero alpha becomes black when converted to pre-multiplied alpha.

It may not be related, but I find that if I take one of the values from the issue, and convert it to RGBa and then back again,

>>> from PIL import Image
>>> im = Image.new("RGBA", (1, 1), (2, 2, 0, 64))
>>> im.convert("RGBa").load()[0, 0]
(1, 1, 0, 64)
>>> im.convert("RGBa").convert("RGBA").load()[0, 0]
(3, 3, 0, 64)

the roundtrip changes the value. That's odd.

@nulano
Copy link
Contributor Author

@nulano nulano commented Mar 27, 2021

It may not be related, but I find that if I take one of the values from the issue, and convert it to RGBa and then back again, [...] the roundtrip changes the value. That's odd.

It's not really that odd when you consider what the operation is doing. Premultiplying with alpha 64, you are rescaling each color from 0-255 to 0-64 range, effectively losing two least significant bits. Therefore there are four different values for each RGBA color band that all map to the same value in RGBa. Converting back to RGBA then chooses the "middle" value.

>>> from PIL import Image
>>> Image.new("RGBA", (1,1), (1,1,0,64)).convert("RGBa").load()[0,0]
(0, 0, 0, 64)
>>> Image.new("RGBA", (1,1), (2,2,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (3,3,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (4,4,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (5,5,0,64)).convert("RGBa").load()[0,0]
(1, 1, 0, 64)
>>> Image.new("RGBA", (1,1), (6,6,0,64)).convert("RGBa").load()[0,0]
(2, 2, 0, 64)
>>> Image.new("RGBa", (1,1), (1,1,0,64)).convert("RGBA").load()[0,0]
(3, 3, 0, 64)

@radarhere
Copy link
Member

@radarhere radarhere commented Mar 27, 2021

Thanks. So that's why NEAREST should not roundtrip through pre-multiply conversion - since it changes the non-alpha channel values.

But have you figured out what the original purpose of it was? In other words, why it's done for any resampling methods? It's not just a shortcut method, as it actually produces different results. I would like to understand the reason so that I know why it's ok to disregard that reason in this case.

@nulano
Copy link
Contributor Author

@nulano nulano commented Mar 27, 2021

The reason to use RGBa is explained by @wiredfool here: #4516 (comment)

When using NEAREST downsampling, the conversion only discards some pixels keeping the ones nearest to the result image grid, while upsampling duplicates pixels.
All other resampling methods use some sort of averaging of a group of pixels near the output pixel, which can cause the colors of transparent pixels to change the colors of non-transparent pixels. See the tests above this one for an example (test_alpha_premult_resize): without premultiplication the resized image would have a partially-transparent gray band where the two regions were touching originally.

Edit: There is also an awesome blog post with excellent animations demonstrating this issue with BILINEAR resampling: http://www.adriancourreges.com/blog/2017/05/09/beware-of-transparent-pixels/

@radarhere
Copy link
Member

@radarhere radarhere commented Mar 27, 2021

All other resampling methods use some sort of averaging of a group of pixels near the output pixel

https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.BOX

PIL.Image.BOX
Each pixel of source image contributes to one pixel of the destination image with identical weights. For upscaling is equivalent of NEAREST.

So that makes BOX an exception, yes?

@nulano
Copy link
Contributor Author

@nulano nulano commented Mar 27, 2021

All other resampling methods use some sort of averaging of a group of pixels near the output pixel

https://pillow.readthedocs.io/en/stable/handbook/concepts.html#PIL.Image.BOX

PIL.Image.BOX
Each pixel of source image contributes to one pixel of the destination image with identical weights. For upscaling is equivalent of NEAREST.

So that makes BOX an exception, yes?

Ah, yes, but only for upsampling. Downsampling produces an average of multiple pixels, with each included pixel having the same weight.

@radarhere radarhere merged commit f799915 into python-pillow:master Mar 28, 2021
50 checks passed
@nulano nulano deleted the 5300 branch Mar 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants