Inefficient alpha compression in lossy JXL

[jendalinda]

When encoding lossy JXL, distance/quality of alpha channel cannot be adjusted separately and depends on the overall distance/quality setting. This may result in inefficient compression of alpha channel and larger file size.

In the example bellow, there are three JXL files. All of them were cretaed using the recent version of cjxl.

1. The lone alpha channel encoded as grayscale image, arguments: "-d 0 -e 9". The file size is 1091 bytes.
2. The RGB image, arguments: "-e 9". The file size is 14861 bytes.
3. The RGB+Alpha image combinng the previous images, arguments: "-e 9". The file size is 19594 bytes.

Apparetly the image 3 is notieably larger than images 1 and 2 added together. The reason is the lossless encoding, in this case, compresses the alpha channel much more efficiently than lossy encoding.

My suggested solution is to allow specifying independent distance/quality for alpha channel. Perhaps the default could be set to lossless alpha. Lossless compression is very efficient for alpha consisting mostly of fully transparent and fully opaque areas.

[jendalinda]

Adding the "-a" option for setting alpha distance is a big improvement. However, RGB+Alpha image is still bigger than RGB and Alpha components encoded as separate images.
Tried "cjxl -d 1 -a 0 -e 9", the result is 17513 bytes.
Is it some overhead or is the compression of extra channels less efficient than plain grayscale image?

[jonsneyers]

Can you share the input image so I can try to understand what's happening here?

[jendalinda]

Sorry I made a mistake, I used RGB pixels that were already lossily compressed. The difference is that fresh pixels produce even bigger file.
These are the same PNG filess that were used to produce the original set of JXL files. RGB pixels, alpha as grayscale and RGB+Alpha combined respectively:

VarDCT compression varies slightly betweeen the versions of libjxl, so I did new tests.

• "cjxl rgb.png rgb.jxl -e 9" results in 14953 bytes
• "cjxl a.png a.jxl -d 0 -e 9" results in 1091 bytes
• "cjxl rgba.png rgba.jxl -e 9" results in 19720 bytes
• "cjxl rgba.png rgba0.jxl -a 0 -e 9" results in 19078 bytes

[jonsneyers]

I think this might be a problem with the combination of e8+ and alpha with --keep_invisible=0 (which is the default when doing lossy for the color image).

Encoding with --keep_invisible=1 leads to a smaller size at e8+.

This doesn't really make sense since the point of not keeping invisible pixels is to compress better. But I'm guessing there is some bug causing e8+ to try to keep the invisible pixels anyway (after they have been turned into a smooth mess by preprocessing), cranking up the adaptive quantization weights to the max.

At e7 and lower this should not be an issue. We'll have to fix the behavior at e8+...

[jendalinda]

So I suppose the invisible pixel handling was broken from the beginning and separate distance setting for alpha helped to discover that.