Skip to content

Add COPY command support for Parquet and ORC #150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 29, 2018
Merged

Add COPY command support for Parquet and ORC #150

merged 5 commits into from
Nov 29, 2018

Conversation

dargueta
Copy link
Contributor

@dargueta dargueta commented Aug 3, 2018

Closes #151

  • Allow choosing Parquet and ORC as load formats (see here).
  • Allow choosing fixed_width as a load format as well for consistency with the others. Enforce the presence of the field widths argument if Formats.fixed_width is selected.

Todos

  • MIT compatible
  • Tests
  • Documentation
  • Updated CHANGES.rst

@dargueta dargueta changed the title Add COPY command support for Parquet and ORC Add COPY command support for Parquet, ORC, and Snappy Aug 4, 2018
@dargueta dargueta changed the title Add COPY command support for Parquet, ORC, and Snappy Add COPY command support for Parquet and ORC Aug 4, 2018
@dargueta
Copy link
Contributor Author

dargueta commented Sep 4, 2018

@graingert or @jklukas you look like the main contributors here. Sorry to be a pain but I just wanted to make sure someone saw this. Thanks!

@graingert
Copy link
Member

Hey there, can you split your Travis/tox changes into another PR, thanks!

@dargueta
Copy link
Contributor Author

dargueta commented Sep 4, 2018

can you split your Travis/tox changes into another PR, thanks!

Done!

@graingert
Copy link
Member

LGTM, can you clean up your commits with git rebase?

@dargueta
Copy link
Contributor Author

dargueta commented Sep 4, 2018

There was a bug in one of my tests I had to fix after your LGTM. Do you mean I should squash the commits, or do git rebase origin/master?

Edit: For some reason I'm suddenly getting this:

dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch upstream
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch origin
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git checkout master
error: pathspec 'master' did not match any file(s) known to git.

This wasn't an issue earlier today, and now I can't rebase off of master with this. Ideas?

@graingert
Copy link
Member

@dargueta I think you want git pull --rebase=interactive upstream master

@dargueta
Copy link
Contributor Author

Okay, cleaned them up now.

@dargueta
Copy link
Contributor Author

I think I've got everything fixed. Any other thoughts?

@dargueta
Copy link
Contributor Author

*bump*

Copy link
Member

@jklukas jklukas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@jklukas jklukas merged commit 9fc8d93 into sqlalchemy-redshift:master Nov 29, 2018
@dargueta dargueta deleted the copy-parquet branch November 29, 2018 22:36
@jklukas
Copy link
Member

jklukas commented Dec 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Parquet and ORC formats to COPY
3 participants