Use zopfli instead of flate2 for Gzip compression
#112
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Cargo deb is also used for one-off deployment, without distribution. Including the --install flag. This is why it has the --fast flag. Can you re-add support for it? (level() function on the compressor) |
|
Oh, sure, I was not aware of that, I can add that back! Would you prefer this flag to fall back to |
|
It's fine to use zopfli if it can be in the same ballpark as fast zlib. |
As mentioned in #100, `cargo-deb` used to use the Zopfli crate for compressing data using the Gzip format, which provides overall smaller results than other general-purpose Gzip encoders that cater to speed more. I think this is a good tradeoff for data meant to be compressed once and distributed or decompressed many times, such as Debian packages. However, the `zopfli` crate API did not support the streaming Gzip encoding that `cargo-deb` moved to. This has changed with its latest v0.8.0 release, bringing it more on par with `flate2`, so that's no longer a reason to not use Zopfli. I ran the available test suite and noticed no regressions due to this change.
Zopfli, even when using fast settings, is still quite slow when compared to `flate2` fast settings, and `gzip` with level 1. Therefore, let's bring back `flate2` for fast mode. While at it, let's also tune Zopfli iterations to be less aggressive by default, which should handle bigger files faster.
|
Thanks! I did some quick benchmarks and noticed that Zopfli with fast settings is still noticeably slower than |
|
Thank you |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As mentioned in #100,
cargo-debused to use the Zopfli crate for compressing data using the Gzip format, which provides overall smaller results than other general-purpose Gzip encoders that cater to speed more. I think this is a good tradeoff for data meant to be compressed once and distributed or decompressed many times, such as Debian packages.However, the
zopflicrate API did not support the streaming Gzip encoding thatcargo-debmoved to. This has changed with its latest v0.8.0 release, bringing it more on par withflate2, so that's no longer a reason to not use Zopfli.I ran the available test suite and noticed no regressions due to this change.