Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace bzip2 with zstd or lz4 #296

Open
zackw opened this issue Nov 7, 2018 · 0 comments
Open

Replace bzip2 with zstd or lz4 #296

zackw opened this issue Nov 7, 2018 · 0 comments

Comments

@zackw
Copy link
Collaborator

zackw commented Nov 7, 2018

bzip2 compression produces more compact files on disk than gzip, but it's also very slow both to compress and to decompress. We should consider switching to a newer compression algorithm. Fast decompression should be a priority. zstd is somewhat faster than bzip2 (especially at decompression) for equivalent compressed sizes, and lz4 is much faster than bzip2 but only about as good as gzip when it comes to compressed sizes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant