New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability for COPY to write directly to gzip/compressed output files #292

Open
RCura opened this Issue Nov 27, 2018 · 0 comments

Comments

Projects
None yet
3 participants
@RCura

RCura commented Nov 27, 2018

NB. This enhancement proposal was first written in the Omnisci community, in this thread.
It is also related to a previously implemented enhancement (#105 )

MapD/Omnisci Core is already using libarchive for inputting compressed files using the COPY * FROM * arguments.
It would be valuable to be also able to export tables to compressed formats, as it would provide an easier way to backup the tables or to transfer those from/to different MapD Core backends.

The main syntax as for the COPY FROM could be used in COPY TO and detect automatically the appropriate compression algorithm based on the filename extension.

Could be written like that for the end user :

COPY (SELECT * FROM tableName) TO 'dataset.csv.tar.gz';
-- OR
COPY (SELECT * FROM tableName) TO 'dataset.csv(.tar).xz';

This would furthemore allow a symetrical handling of compressed files :

COPY (SELECT * FROM tableName) TO 'dataset.csv.tar.gz';
-- THEN
TRUNCATE TABLE tableName;
COPY tableName FROM 'dataset.csv.tar.gz';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment