Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Zstandard support for zip archives and CHDs #11827

Merged
merged 4 commits into from Dec 10, 2023
Merged

Conversation

cuavas
Copy link
Member

@cuavas cuavas commented Dec 9, 2023

This does three things:

  • Adds the Zstandard library in 3rdparty with an option to use the system-wide installation (dual BSD and GPL license).
  • Adds support for zip archives members compressed with Zstandard (method 93).
  • Adds support for Zstandard-compressed hunks in CHDs (zstd and cdzs codecs).

We occasionally get requests for Zstandard compression support (e.g. issue #7402). It gives far better decompression performance than zlib or particularly LZMA while delivering better compression ratios than zlib. This is important to people using emulators on low-end devices. CHD decompression speed affects performance of emulated software that streams from hard disk or CD, and zip decompression performance affects initial load time for emulated systems.

By default, chdman will not use Zstandard compression. This ensures that new CHDs will be backwards compatible if the user does not intentionally enable Zstandard compression.

You can create zip archives using Zstandard compression with WinZip, a fork of p7zip, and various other tools.

* Fast x86-64 assembly language Huffman decoder disabled for now.
* Only compression and decompression components built.
* Library documentation not included.
Zstandard is not enabled by default so new CHDs will not be incompatible
by default.  Use zstd for regular Zstandard codec, and cdzs for Zstandard
CD codec.
@cuavas
Copy link
Member Author

cuavas commented Dec 9, 2023

The main impact this has it that it bloats the source a bit. Apart from that, it’s pretty much invisible to the user if they don’t consciously try to use it.

@cuavas cuavas merged commit 05e69b4 into mamedev:master Dec 10, 2023
3 of 5 checks passed
@cuavas cuavas deleted the zstd branch December 10, 2023 23:48
@crashGG
Copy link

crashGG commented Dec 11, 2023

Finally, chd has added zstd support. If used properly, zstd decompression will exceed the speed of copying.

Luigi30 pushed a commit to Luigi30/mame that referenced this pull request Dec 16, 2023
* 3rdparty/zstd: Added Zstandard compression library version 1.5.5.
* util/unzip.cpp: Added support for Zstandard compression (method 93).
* util/chdcodec.cpp: Added support for Zstandard compression.
* 3rdparty/flac: Always define NDEBUG to avoid log spam.
@saulfabregwiivc
Copy link

This seems promising, maybe we can run these CHDs in Wii soon :)

einstein95 pushed a commit to einstein95/mame that referenced this pull request Mar 2, 2024
* 3rdparty/zstd: Added Zstandard compression library version 1.5.5.
* util/unzip.cpp: Added support for Zstandard compression (method 93).
* util/chdcodec.cpp: Added support for Zstandard compression.
* 3rdparty/flac: Always define NDEBUG to avoid log spam.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants