-
Notifications
You must be signed in to change notification settings - Fork 322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rebalance compression levels. #31
Conversation
1243b10
to
2d8c5f7
Compare
One thing I note about the original values: they seem to correspond to the compression levels in zlib. Even the I like the idea of having more speed/size tradeoffs available, but I'd also like a note in the docs saying that the compression levels work differently than zlib. |
@willglynn You are right. It makes good sense to document why these values are different than zlib. That said, in my tests zlib has always performed ~1 level worse than go deflate in terms of compression. Noting that you shouldn't expect settings to carry over 1:1 makes sense. |
This will rebalance the compression levels to have a more even distribution of compression levels. The general trend of this is to make compression faster, which means that a given compression level (except 1 and 9) will be faster. The existing implementation favors compression, and from level 5 and up, very small compresssion improvements are added for very high computation cost. Rough guidelines: Previous level 3 speed is now equal to level 6. New default is level 5, which in most cases offers a quite good compression, without getting diminished returns from additional computation power. See "rebalance" tabs for benchmarks of rebalanced levels: https://docs.google.com/spreadsheets/d/1nuNE2nPfuINCZJRMt6wFWhKpToF95I47XjSsc-1rbPQ/edit?usp=sharing Make level 6 slightly faster, the decreased speed didn't yeild significant improvements. Create new level 2 encoder. Speed up level 1 by reusing the table. Add documentation. Remove buffer, if we get small block. Update documentation Add Level 3 "Snappy" compressor, which tests up to two hashes.
9439634
to
5f70209
Compare
Rebalance compression levels.
This will rebalance the compression levels to have a more even distribution of compression levels. The general trend of this is to make compression faster, which means that a given compression level (except 1 and 9) will be faster.
The existing implementation favors compression, and from level 5 and up, very small compresssion improvements are added for very high computation cost.
Rough guidelines: Previous level 3 speed is now equal to level 6.
New default is level 5, which in most cases offers a quite good compression, without getting diminished returns from additional computation power.
See "rebalance" tabs for benchmarks of rebalanced levels:
https://docs.google.com/spreadsheets/d/1nuNE2nPfuINCZJRMt6wFWhKpToF95I47XjSsc-1rbPQ/edit?usp=sharing