Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--compress-program option for xsv split? #303

Open
d33tah opened this issue Jun 23, 2022 · 4 comments
Open

--compress-program option for xsv split? #303

d33tah opened this issue Jun 23, 2022 · 4 comments

Comments

@d33tah
Copy link

d33tah commented Jun 23, 2022

Today I learned how fast xsv split can be. The only thing that I feel that it's missing is a --compress-program switch (or similar) that would allow the user to compress created split files. @BurntSushi do you feel like it's a good fit for this tool?

@BurntSushi
Copy link
Owner

Nope. It seems totally orthogonal to me. You can always compress them after the split command.

@d33tah
Copy link
Author

d33tah commented Jun 23, 2022

@BurntSushi Thanks for replying! The problem is that you need to store files in the uncompressed form in the meantime. Which, for huge CSV files - which benefit from splitting the most - can be quite cumbersome. In my case I decided to mount a compressed btrfs system to satisfy xsv's requirements, but having a way to compress data before saving to the disk would IMO solve the problem better.

@LemmingAvalanche
Copy link

@d33tah See qsv snappy.

@d33tah
Copy link
Author

d33tah commented Jun 27, 2023

@LemmingAvalanche thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants