XZ on QOI is better than zopflipng

[Glandos]

I am always curious of data compression.
I tried your corpus, and compare with zopflipng. What surprised me a lot is that QOI + XZ is smaller than zopflipng. XZ is run with -9 and zopflipng with --prefix -m

• Kodak set

tests/images/kodak on  master [?] ❯ du -c kodim*.png | tail -1
15072	total
tests/images/kodak on  master [?] ❯ du -c zopfli_kodim*.png | tail -1
14424	total
tests/images/kodak on  master [?] ❯ du -c kodim*.qoi | tail -1
18568	total
tests/images/kodak on  master [?] ❯ du -c kodim*.qoi.xz | tail -1
13760	total
tests/images/kodak on  master [?] ❯ du -c zopfli_kodim*.png.xz | tail -1
14424	total

• Screenshot set is much more impressive

tests/images/screenshots on  master [?] ❯ find -name '*.png' ! -name 'zopfli_*' -print0 | xargs -0 du -c | tail -1
33216	total
tests/images/screenshots on  master [?] ❯ du -c zopfli_*.png | tail -1
21912	total
tests/images/screenshots on  master [?] ❯ du -c * | tail -1
128364	total
tests/images/screenshots on  master [?] ❯ du -c *.qoi | tail -1
33600	total
tests/images/screenshots on  master [?] ❯ du -c *.qoi.xz | tail -1
18132	total
tests/images/screenshots on  master [?] ❯ du -c zopfli*.xz | tail -1
21504	total

Is there any explanation on why it can beat out PNG like this?

[sbrki]

From PNG spec:

The encoder is allowed to change filters for each new scanline.

This means the for a single row, only one kind of filtering is applied with PNG. QOI effectively chooses optimal "filter" for each subsequent pixel, which I assume gives better compression ratio.

On the other hand, it is possible that QOI somehow produces lower entropy output if you think of a QOI block as a dictionary entry, thus making the job for xz easier, but I'm not sure if that is the case (or why).

[roytam1]

What surprised me a lot is that QOI + XZ is smaller than zopflipng.

this depends on sample. I have a scanned document sample that QOI loses a lot. WebP lossless and JPEG2000 lossless don't win a lot than PNG.

[Glandos]

Did you try xz -9 on Scan.qoi ?

[roytam1]

Did you try xz -9 on Scan.qoi ?

sure

H:\Temp>timer convert Scan.png -define webp:lossless=true Scan.webp
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.280 =    0%
User Time    =    47.923 =   99%
Process Time =    48.204 =   99%
Global Time  =    48.306 =  100%

H:\Temp>timer convert Scan.png -quality 0 Scan.jp2
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.343 =    3%
User Time    =     9.016 =   96%
Process Time =     9.360 =   99%
Global Time  =     9.360 =  100%

H:\Temp>timer qoiconv.exe Scan.png Scan.qoi
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.078 =    8%
User Time    =     0.748 =   82%
Process Time =     0.826 =   90%
Global Time  =     0.910 =  100%

H:\Temp>timer c:\cygwin-1.7\bin\xz.exe -k -9 Scan.qoi
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.218 =    0%
User Time    =    24.804 =   98%
Process Time =    25.022 =   99%
Global Time  =    25.078 =  100%

H:\Temp>dir/os scan*
 Volume in drive H is 
 Volume Serial Number is 

 Directory of H:\Temp

30/11/2021  16:59        23,335,272 Scan.jp2
30/11/2021  16:59        23,413,534 Scan.webp
28/11/2021  07:23        23,988,771 Scan.png
30/11/2021  16:59        24,633,436 Scan.qoi.xz
30/11/2021  16:59        36,217,424 Scan.qoi
               5 File(s)    131,588,437 bytes
               0 Dir(s)

[Glandos]

OK, thanks for the test. It seems that QOI + XZ can perform well where PNG is currently good: screenshots or pictures with large identical colored zones.