Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make parquet compression options a flag to pass in to benchmark #7

Open
cholmes opened this issue Jul 26, 2023 · 0 comments
Open

Make parquet compression options a flag to pass in to benchmark #7

cholmes opened this issue Jul 26, 2023 · 0 comments
Labels
benchmarking issues related to benchmarking performance

Comments

@cholmes
Copy link
Collaborator

cholmes commented Jul 26, 2023

Description

There are a number of different parquet compression options (snappy, gzip, zstd, brotli, uncompressed, etc) that can make things faster/slower and smaller/bigger. It'd be nice to be able to benchmark / compare those. Right now there's a global variable that can be used to set this. If implemented it should raise appropriate errors on the process used, as each process supports a different set of compression options.

@cholmes cholmes changed the title Make parquet compression options a flag to pass in Make parquet compression options a flag to pass in to benchmark Aug 31, 2023
@cholmes cholmes added the benchmarking issues related to benchmarking performance label Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarking issues related to benchmarking performance
Projects
None yet
Development

No branches or pull requests

1 participant