Format parquet doesn't support any compression codec #5865
Assignees
Labels
enhancement
New feature or request
Comments
kewang1024
added a commit
to kewang1024/velox
that referenced
this issue
Jul 27, 2023
…#5795) Summary: As of now, velox use config to decide compression type for write. We want the ability to use the dynamically passed-in compression type by application. In this PR, we introduce compressionKind in HiveInsertTableHandle to give applications the ability to do so, if it's not passed in, we use the config compressionKind by default. Tests on parquet writer using compression would be added once facebookincubator#5796 and facebookincubator#5865 are resolved Pull Request resolved: facebookincubator#5795 Reviewed By: xiaoxmeng Differential Revision: D47819287 Pulled By: kewang1024 fbshipit-source-id: 4d0b8cdfd79c1d16651a7140619e1c937e5056d4
kewang1024
added a commit
to kewang1024/velox
that referenced
this issue
Jul 27, 2023
…#5795) Summary: As of now, velox use config to decide compression type for write. We want the ability to use the dynamically passed-in compression type by application. In this PR, we introduce compressionKind in HiveInsertTableHandle to give applications the ability to do so, if it's not passed in, we use the config compressionKind by default. Tests on parquet writer using compression would be added once facebookincubator#5796 and facebookincubator#5865 are resolved Pull Request resolved: facebookincubator#5795 Reviewed By: xiaoxmeng Differential Revision: D47819287 Pulled By: kewang1024 fbshipit-source-id: a5fdbd1c72dfa7d81b0bd70ab8246012ae2cb7f0
kewang1024
added a commit
to kewang1024/velox
that referenced
this issue
Jul 27, 2023
…#5795) Summary: As of now, velox use config to decide compression type for write. We want the ability to use the dynamically passed-in compression type by application. In this PR, we introduce compressionKind in HiveInsertTableHandle to give applications the ability to do so, if it's not passed in, we use the config compressionKind by default. Tests on parquet writer using compression would be added once facebookincubator#5796 and facebookincubator#5865 are resolved Pull Request resolved: facebookincubator#5795 Reviewed By: xiaoxmeng Differential Revision: D47819287 Pulled By: kewang1024 fbshipit-source-id: de944306339eef12766c838d3bb27efff3d3c7a9
facebook-github-bot
pushed a commit
that referenced
this issue
Jul 28, 2023
Summary: As of now, velox use config to decide compression type for write. We want the ability to use the dynamically passed-in compression type by application. In this PR, we introduce compressionKind in HiveInsertTableHandle to give applications the ability to do so, if it's not passed in, we use the config compressionKind by default. Tests on parquet writer using compression would be added once #5796 and #5865 are resolved Pull Request resolved: #5795 Reviewed By: xiaoxmeng Differential Revision: D47819287 Pulled By: kewang1024 fbshipit-source-id: 771759d6b83e7c48d0dd198b6ea0d64b8224dcab
|
This is likely out of date. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Add support to parquet to support compression
The text was updated successfully, but these errors were encountered: