[Qt] optimize PNG files

[jonasschnelli]

According to report on #5488 there can be libpng warnings while starting bitcoin-qt.
This PR will strip out unnecessary data (especially color profiles) from all PNG files.

[laanwj]

Can you give the exact command line used? I'd like to reproduce this.

[jonasschnelli]

i used convert -strip <filename>.png <filename>.png.

[paveljanik]

Doesn't print the warning as the master itself.
For the reference, the warning was:

libpng warning: iCCP: known incorrect sRGB profile

[sipa]

The result of that command is not identical here. Running the command on top of your output gives an even different result.

[jonasschnelli]

Analyzed.
It needs two (or even three runs). After then that file SHA1 keep the same.
I also provided a script under ./contrib/strip_pngs.sh

I also assume it could depend on installed imagemagick and libpng version.

Here you can see the process of one file.
The DIFF is from imagemagicks identify -verbose
(full bash dump of all files is here: http://paste.ubuntu.com/9548096/)

Jonass-MacBook-Pro-2:bitcoin jonasschnelli$ convert --version
Version: ImageMagick 6.8.9-8 Q16 x86_64 2014-10-23 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC Modules
Delegates: bzlib freetype jng jpeg ltdl lzma png xml zlib

Jonass-MacBook-Pro-2:bitcoin jonasschnelli$ contrib/strip_pngs.sh src/qt/res/icons
stripping ./about.png
SHA1 before: 
SHA1(./about.png)= e6f23d98c69892f0a26613c23a68a610030f745b
SHA1 after: 
SHA1(./about.png)= 8b56adeec521bba7bc6d49e7dc54f96216af0f7f

DIFF:
8c8
<   Base type: GrayscaleAlpha
---
>   Base type: Grayscale
532,538c532,533
<   Rendering intent: Perceptual
<   Gamma: 0.454545
<   Chromaticity:
<     red primary: (0.64,0.33)
<     green primary: (0.3,0.6)
<     blue primary: (0.15,0.06)
<     white point: (0.3127,0.329)
---
>   Rendering intent: Undefined
>   Gamma: 1
544c539
<   Intensity: Undefined
---
>   Intensity: Rec709Luminance
552,553c547,550
<     date:create: 2014-12-17T08:51:59+01:00
<     date:modify: 2014-12-17T08:51:59+01:00
---
>     date:create: 2014-12-17T08:52:20+01:00
>     date:modify: 2014-12-17T08:52:20+01:00
>     png:bKGD: chunk was found (see Background color, above)
>     png:cHRM: chunk was found (see Chromaticity, above)
556,557c553,554
<     png:IHDR.color-type-orig: 6
<     png:IHDR.color_type: 6 (RGBA)
---
>     png:IHDR.color-type-orig: 4
>     png:IHDR.color_type: 4 (GrayAlpha)
560,561d556
<     png:sRGB: intent=0 (Perceptual Intent)
<     png:text: 1 tEXt/zTXt/iTXt chunks were found
563d557
<     Software: Adobe ImageReady
567,568c561,562
<   Tainted: True
<   Filesize: 5.92KB
---
>   Tainted: False
>   Filesize: 4.35KB

Jonass-MacBook-Pro-2:bitcoin jonasschnelli$ contrib/strip_pngs.sh src/qt/res/icons
stripping ./about.png
SHA1 before: 
SHA1(./about.png)= 8b56adeec521bba7bc6d49e7dc54f96216af0f7f
SHA1 after: 
SHA1(./about.png)= 4a7719bc47e073de262fddf940e774712d999c8e

DIFF:
547,548c547,548
<     date:create: 2014-12-17T08:52:20+01:00
<     date:modify: 2014-12-17T08:52:20+01:00
---
>     date:create: 2014-12-17T08:52:34+01:00
>     date:modify: 2014-12-17T08:52:34+01:00
550c550
<     png:cHRM: chunk was found (see Chromaticity, above)
---
>     png:gAMA: gamma=1 (See Gamma, above)
562c562
<   Filesize: 4.35KB
---
>   Filesize: 4.32KB


Jonass-MacBook-Pro-2:bitcoin jonasschnelli$ contrib/strip_pngs.sh src/qt/res/icons
stripping ./about.png
SHA1 before: 
SHA1(./about.png)= 4a7719bc47e073de262fddf940e774712d999c8e
SHA1 after: 
SHA1(./about.png)= 4a7719bc47e073de262fddf940e774712d999c8e

DIFF:
547,548c547,548
<     date:create: 2014-12-17T08:52:34+01:00
<     date:modify: 2014-12-17T08:52:34+01:00
---
>     date:create: 2014-12-17T08:52:42+01:00
>     date:modify: 2014-12-17T08:52:42+01:00

[laanwj]

OK this freaks me out. My output differs from yours. From your diff above I thought the problem was an embedded timestamp, however I get no different result from

    export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1
    export FAKETIME='2012-12-12 12:12:12'
    git reset --hard
    find -name \*.png -print0 | xargs -0 -I FILENAME convert -strip FILENAME FILENAME
    find -name \*.png -print0 | xargs -0 sha256sum > /tmp/a.txt

As from

    unset LD_PRELOAD
    unset FAKETIME
    git reset --hard
    find -name \*.png -print0 | xargs -0 -I FILENAME convert -strip FILENAME FILENAME
    find -name \*.png -print0 | xargs -0 sha256sum > /tmp/b.txt

As it seems, identify.im6 just shows the current timestamp as date:create and date:modify. Confusing. But indeed I get the same output every time: http://paste.ubuntu.com/9550689/

So probably it's just a version difference. My IM is much older:

Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org

Edit: OH, I'm running it only one time, not until fixpoint. Will try.

[laanwj]

Now ran it three times, hash was not longer changing. However, still not all match the files in this pull (though some do). See diff here http://paste.ubuntu.com/9550744/

[jonasschnelli]

I assume this happens because of different imagemagick/libpng versions. I mean, the core business of these tools is to deal with png/bitmap internas. So there might be some byte differences because of different handling.
Adding imagemagick and libpng to the depends would probably solve this but is a overkill IMO.

[laanwj]

Yes, it's not that important. I primarily wanted to be able to reproduce this when people add files (which the script does). Determinism would be a plus, but adding imagemagick to depends would be overkill.

[fanquake]

Needs Rebase

[jonasschnelli]

Rebased.

[jonasschnelli]

Updated the script as well as the images so it will use pngcrush to crush the convert-stripped images.

[zander]

Would a tool like optipng or pngcrush or trimage be more appropriate ?

[jonasschnelli]

@zander: pngcrush is included. See my last comment.

[laanwj]

Your script needs set -e at the top to make it fail when one of the tools fails.
(e.g. I didn't have pngcrush installed and it happily continued)

Also let's move it to contrib/devtools, leaving the script in the contrib toplevel is a bit untidy.

[jonasschnelli]

Rewrote script, placed it under contrib/devtools.
Output of processing is here: https://gist.github.com/jonasschnelli/dad302d9b9b8a3163259

[jonasschnelli]

A second run of the script won't change the PNG sha256 (at least in my environment).

[laanwj]

Looks good to me now. When I re-run this script I get the same output as well.
In laanwj@41dc934 I've improved the script a little bit:

• Add pre/post check of image contents
• remove use of external tool to compute sha256 in favor of hashlib

[laanwj]

Added another commit laanwj@2c77c15 that replaces uses of shell=True with the Python equivalent (as shell=True is less portable, and can be a security hazard).

[jonasschnelli]

Nice. Thanks for the additions.
I also added a rename commit because strip_png.py is no longer a adequate name.

[laanwj]

Merged via d1aa3c6 (squashed into two commits, one with your changes one with mine)

[jonasschnelli]

Was merged. Closing.