You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are a number of different parquet compression options (snappy, gzip, zstd, brotli, uncompressed, etc) that can make things faster/slower and smaller/bigger. It'd be nice to be able to benchmark / compare those. Right now there's a global variable that can be used to set this. If implemented it should raise appropriate errors on the process used, as each process supports a different set of compression options.
The text was updated successfully, but these errors were encountered:
cholmes
changed the title
Make parquet compression options a flag to pass in
Make parquet compression options a flag to pass in to benchmark
Aug 31, 2023
Description
There are a number of different parquet compression options (snappy, gzip, zstd, brotli, uncompressed, etc) that can make things faster/slower and smaller/bigger. It'd be nice to be able to benchmark / compare those. Right now there's a global variable that can be used to set this. If implemented it should raise appropriate errors on the process used, as each process supports a different set of compression options.
The text was updated successfully, but these errors were encountered: