-
-
Notifications
You must be signed in to change notification settings - Fork 99
Enable configuration of the zstd compression level for feather store #2623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I guess I still need to add a changelog entry, extend NB, there is no documentation for parquet plugin configuration options. Where's a good place for that? |
I'd say under tuning: https://vast.io/docs/setup/tune |
18f2bd5
to
1c99ee3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is almost there. Happy to see this lands, especially in light of the data analysis from the recent blog post.
In favor of plugin-specific Zstandard compression level.
d92f564
to
563d742
Compare
0297a3f
to
e513569
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just wordsmithing now. I think this is ready to go! 🚀
Co-authored-by: Dominik Lohmann <mail@dominiklohmann.de>
This came up during work on the parquet / feather benchmarks, where we want to measure the effect of various compression levels for both parquet and feather.
This change follows closely what we do for the parquet plugin to make the zstd compression level configurable.
📝 Reviewer Checklist
Review this pull request by ensuring the following items: