Skip to content

Support more indexed PNGs and compress PNGs#21138

Merged
Mailaender merged 4 commits into
OpenRA:bleedfrom
RoosterDragon:png-compress
Nov 25, 2023
Merged

Support more indexed PNGs and compress PNGs#21138
Mailaender merged 4 commits into
OpenRA:bleedfrom
RoosterDragon:png-compress

Conversation

@RoosterDragon

Copy link
Copy Markdown
Member

Teach PNG decoder to handle indexed bit depths of 1, 2 or 4.

The PNG decoder, when dealing when indexed images with a palette, could only decode a bit depth of 8. Teach it to decode depths of 1, 2 and 4 as well. As the palette data is exposed to consumers of the PNG class, unpack the data into a 8 bit depth so consumers don't need to also handle the new bit depths.


Add UnpackMapCommand

This command allows either unpacking oramap files into folders, or packing folders into oramap files.

Example invocations:
"d2k --unpack-map unpack" to unpack maps of the d2k mod into folders.
"cnc --unpack-map repack" to repack maps of the cnc mod into oramap files (but will only pack folders that were unpacked previously).


(See also #20911)

Compress all pngs, including within oramap files.

Reduces size used for png files from 13,366,660 bytes to 13,055,285 bytes in total. Changes size used for oramap files from 2,601,027 bytes to 2,605,783 bytes in total (contained PNGs are smaller, but the oramap zip wrapper didn't compress as well). This slight filesize improvement doesn't noticeably impact loading times.

zopfilpng is used for compression with the following command line:
'zopflipng.exe -y -m image.png image.png'

This follows on from 78bef8a and bc5e7d1. Except now that the PNG decoder supports bit depths of 1, 2 or 4 we don't have to preserve the original bit depth of the image, allowing for more compression.

The oramap files were updated by:

  • Running utility command " --unpack-map unpack" for each mod.
  • Compressing the png files using the command above.
  • Running utility command " --unpack-map repack" for each mod, except in Map.Save the line if (!LockPreview) { var previewData = ... is replaced with if (false) { var previewData = ... to save the existing optimized image on disk rather than generating a fresh preview.

This will make #20918 easier if we employ an external library, as the restriction on bit depths will no longer be required.

@anvilvapre anvilvapre left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps test also using an external image converter.

submitting the code independant of the pngs would have been clearer.

Comment thread OpenRA.Game/FileFormats/Png.cs
Comment thread OpenRA.Game/FileFormats/Png.cs
@RoosterDragon

Copy link
Copy Markdown
Member Author

perhaps test also using an external image converter.

What sort of test do you envision?

submitting the code independant of the pngs would have been clearer.

The code changes and file changes are in separate commits.

@anvilvapre

Copy link
Copy Markdown
Contributor

perhaps test also using an external image converter.

What sort of test do you envision?

generate png with i.e. imagemagic and read them.
write png and read them with imagemagic.
or any external image tool.
not as a unit test.

likely you already did this.

@RoosterDragon

Copy link
Copy Markdown
Member Author

The compressed images are the test case - that's why I've included them in this PR! The compressed images in this PR rely on the new code that handles bit depths - they wouldn't load without the code changes.

@anvilvapre

Copy link
Copy Markdown
Contributor

i only now read that you generated the png files with an external tool 'zopfilpng'. where i had noticed the repack command source and my assumption was that you also had written an encoder and had compressed them with your own code. i should have read the issue comment.

@pchote pchote left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, just a couple of comments.

Do you plan a followup that changes Map.SavePreview to save indexed png? (if not, I may do it).

Comment thread OpenRA.Game/FileFormats/Png.cs Outdated
Comment thread OpenRA.Mods.Common/UtilityCommands/UnpackMapCommand.cs Outdated
@RoosterDragon

Copy link
Copy Markdown
Member Author

Do you plan a followup that changes Map.SavePreview to save indexed png? (if not, I may do it).

See #20918, @Bujacikk may already be working on something there.

The PNG decoder, when dealing when indexed images with a palette, could only decode a bit depth of 8. Teach it to decode depths of 1, 2 and 4 as well. As the palette data is exposed to consumers of the PNG class, unpack the data into a 8 bit depth so consumers don't need to also handle the new bit depths.
This command allows either unpacking oramap files into folders, or packing folders into oramap files.

Example invocations:
"d2k --unpack-map unpack" to unpack maps of the d2k mod into folders.
"cnc --unpack-map repack" to repack maps of the cnc mod into oramap files (but will only pack folders that were unpacked previously).
Reduces size used for png files from 13,366,660 bytes to 13,055,285 bytes in total. Changes size used for oramap files from 2,601,027 bytes to 2,605,779 bytes in total (contained PNGs are smaller, but the oramap zip wrapper didn't compress as well). This slight filesize improvement doesn't noticeably impact loading times.

zopfilpng is used for compression with the following command line:
'zopflipng.exe -y -m image.png image.png'

This follows on from 78bef8a and bc5e7d1. Except now that the PNG decoder supports bit depths of 1, 2 or 4 we don't have to preserve the original bit depth of the image, allowing for more compression.

The oramap files were updated by:
- Running utility command "<mod> --unpack-map unpack" for each mod.
- Compressing the png files using the command above.
- Running utility command "<mod> --unpack-map repack" for each mod, except in Map.Save the line `if (!LockPreview) { var previewData = ...` is replaced with `if (false) { var previewData = ...` to save the existing optimized image on disk rather than generating a fresh preview.
This provides a single utility command for interacting with maps, that takes an arg for the map operation. The filename filter allows all maps in the mod to be operated on by default, or a regex can be passed to limit the operation to certain maps.
@Mailaender Mailaender merged commit 6b0db66 into OpenRA:bleed Nov 25, 2023
@Mailaender

Copy link
Copy Markdown
Member

Changelog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants