New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make export compression useful or get rid of it entirely #4446
Comments
I'm fine with these suggestions. I only think we need to keep some versions being able to still read the current format, right? |
Not really, import/export is only compat between Z streams |
As far as I'm aware, we only allow exports to be imported on the same Y stream they were produced on. So the only rule is that we can't remove compatibility with the gzipped export files from existing releases - but we can remove it in new releases, because those new releases just need to be able to import whatever format they're currently exporting. |
It appears that Gzip at all (even using level 0 as we do) still incurs some overhead during imports, on the order of 12% or so in the test I did. It's calculating a lot of CRC32 checksums internally, even though we are verifying integrity ourselves, and also it appears that it still needs to go through some "decompression" routines to seek around within the file. |
see: #4434
We could do any of the following:
This would be a forwards-only change, no backports.
The text was updated successfully, but these errors were encountered: