Premultiply by default on png

[ghost]

#794

This turns on premultiply by default for png files.

[RodneyBaker]

I'm certainly glad this can be changed.

I understand the panic that sets in when people see images with the halo around it.
While the math behind the scenes is what is most important most people won't appreciate the difference but it can help to understand that "premultiplied" alpha is actually polluting the color channels. With straight Alpha the channels are optimal (each individually) and the end result depends on the math and compositing processes applied to each.

I find it interesting that Alpha Channels have increasingly become similar to Gamma workflow in that the source and target must be considered carefully. This is likely the result of digital artists using a wider range of programs than they did in the past; where in those days they primarily stayed within the graphical environment of one application where the type of math pre-applied to RGBA channels never changed.

Perhaps the oddest thing is that using premultiplied images can force a user to later remove the color matting from the color channels (i.e. unpremultiply them) to get back at cleaner color channels (because the RGB channels have been inadvertently altered by the act of premultiplying their values with the alpha channel. Whether they can do this will depend on the program they use and the options available to them. Fortunately, some programs are being taught how to recognize whether images are straight or premultiplied out of the gate (upon import). At a guess I'd say the image is analyzed for sharp drop offs (hard edges) in the color channels and where predominantly found are interpreted as Alpha Straight?

I've always found Alpha Channels to be fascinating (and powerful) but I had no idea just how fascinating they were.

[shun-iwasawa]

LGTM Thank you @turtleTooth !

@RodneyBaker In MotionBlurFxIwa, I implemented the procedure for judging whether the source image is premultiplied or straight alpha here.
For the premultiplied image, each RGB channel value must be equal to or less than Alpha channel value for every pixels. So if you find at least one pixel of which R/G/B channel is greater than its alpha channel, the image can be guessed as straight alpha.

[RodneyBaker]

@shun-iwasawa said:

@RodneyBaker In MotionBlurFxIwa, I implemented the procedure for judging whether the source image is premultiplied or straight alpha here. For the premultiplied image, if channels are normalized to 0.0 - 1.0, each RGB channel value must be equal to or less than Alpha channel value for every pixels. So if you find at least one pixel of which R/G/B channel is greater than its alpha channel, the image can be guessed as straight alpha.

Very interesting.I think I actually understood that!
For speed of processing I find myself wondering if your test is running a sample across the middle of the image first, starting from the top/left, or some other approach. It is useful that only one pixel needs to be found where R/G/B is greater than it's alpha to identify the type of alpha. Nice.

Thank you for the information and I look forward to trying out MotionBlurFXIwa!

Added: I conducted a small test with two images rendered out of another program (one premultiplied and one straight) and the results were interesting but I haven't quite got to a conclusion. If I import an image into the xsheet that is already premultiplied and OpenToonz is set to premultiply it appears that an extra multiplication is conducted that creates a slightly dark/sharp line around the image. Whereas, the unpremultiplied image is premultiplied upon import and has a soft edge (the ideal edge in most cases I would think). My first impression is that something isn't quite right here but I'm not exactly sure what it is.

I think the default setting is going to cause some confusion because the assumption was that users would mostly be importing premultiplied images. However, if they are straight then they look correct but if premultiplied they gain another premultiplication which causes the result not to be correct. This is easily corrected either way (by turning the premultiply setting on or off in the Level Settings) but I am finding the results curious. I'll see if I an share my results in a way that others will understand.

As errors go... I see where the black border is preferrable to the white halo so the new default setting of premultiplying for PNGs would appear to be the better of the two but we may need to look again to make sure a second pass of premultiplication isn't taking an already 'correct' image and adding the dark border. That wouldn't be ideal.

I need to test with some images saved out of Photoshop, After Effects etc.

[RodneyBaker]

I'm going to go out on a limb and suggest something isn't quite right in the processing of premultiplied PNGs.

Assumption: OpenToonz will render out images with premultiplied alphas.
I don't see an option to have OT render "straight" alphas although I assume there must be.

Try this test:
Render out a PNG image from OpenToonz that has an alpha channel. For instance, use the Type Tool to create some text.

Added: You'll likely have to add a blur via Schematic Blur node in order to set up the test properly for observing any halo around the text. I set the blur to 2 or 3.

Import/Load that PNG into the xsheet and then Right Click on that frame/level in the xsheet to select Level Settings where we will toggle on/off the "premultiply" setting.
Test with premultiply on and render a preview in OT.
Note the dark edge around the text.
Test with premultiply off and render a preview in OT.
Note the edge is smooth (no dark edge).

My assumpion is that the alpha channel is being multiplied twice but I may be wrong in this. Hopefully I'm right as this might be an esay fix.

Of note (as mentioned in my post above) I tested with a 'straight' alpha rendered in another program and the image looks correct with "premultiply" on in Level Settings and wrong with it off (the image has the dreaded white halo. So, OT is handing "straight" alphas with PNG images correctly by default. So that much appears to be correct.

This is with the latest nightly build as of 2017-05-26 08:10:40 UTC.

[shun-iwasawa]

@RodneyBaker Thank you for pointing the problem out !

The problem is that OpenToonz renders the PNG image with premultiplied alpha.

According to the PNG Specification , it says that PNG should be straight-alpha in normal.

The colour values in a pixel are not premultiplied by the alpha value assigned to the pixel. This rule is sometimes called "unassociated" or "non-premultiplied" alpha. (Another common technique is to store sample values premultiplied by the alpha value; in effect, such an image is already composited against a black background. PNG does not use premultiplied alpha. In consequence an image editor can take a PNG image and easily change its transparency.)

[shun-iwasawa]

I posted a new issue.

[RodneyBaker]

Thanks for the quick response and posting of the issue. :)

For completeness sake...
I tested with the TIF format as well and see the same result as with PNG.

I also tested TGA.
The TGA format has a few other considerations but appears to be the same as PNG and TIF. As the Alpha Channel is only supported in the 32bit flavor of TGA the abscense of an alpha channel in 8 and 24bit flavors might confuse some users BUT... TGA isn't a commonly used format these days except by those who will likely know how to handle it. After rall these years... TGA is still a great format.

[shun-iwasawa]

PNG supports straight alpha only whereas TIFF and TGA also support premultiplied alpha.
Applying "premultiply" option to already-premultiplied image will cause dark edge due to unnecessary double-multiplication.

In Japanese animation industry they are handling tons of TGA files as it is standard format for RETAS PaintMan.