Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add COPY command support for Parquet and ORC #150

Merged
merged 5 commits into from Nov 29, 2018

Conversation

@dargueta
Copy link
Contributor

commented Aug 3, 2018

Closes #151

  • Allow choosing Parquet and ORC as load formats (see here).
  • Allow choosing fixed_width as a load format as well for consistency with the others. Enforce the presence of the field widths argument if Formats.fixed_width is selected.

Todos

  • MIT compatible
  • Tests
  • Documentation
  • Updated CHANGES.rst

@dargueta dargueta force-pushed the dargueta:copy-parquet branch 2 times, most recently from 0c3e29e to 9c62cdf Aug 3, 2018

@dargueta dargueta changed the title Add COPY command support for Parquet and ORC Add COPY command support for Parquet, ORC, and Snappy Aug 4, 2018

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 4196b3a to 48c48f9 Aug 4, 2018

@dargueta dargueta changed the title Add COPY command support for Parquet, ORC, and Snappy Add COPY command support for Parquet and ORC Aug 4, 2018

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 068e3a9 to e95afbe Aug 4, 2018

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

@graingert or @jklukas you look like the main contributors here. Sorry to be a pain but I just wanted to make sure someone saw this. Thanks!

@graingert

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2018

Hey there, can you split your Travis/tox changes into another PR, thanks!

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 6cd48bb to 7526e35 Sep 4, 2018

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

can you split your Travis/tox changes into another PR, thanks!

Done!

@graingert

This comment has been minimized.

Copy link
Collaborator

commented Sep 4, 2018

LGTM, can you clean up your commits with git rebase?

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 6104b0b to 21e5f23 Sep 4, 2018

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Sep 4, 2018

There was a bug in one of my tests I had to fix after your LGTM. Do you mean I should squash the commits, or do git rebase origin/master?

Edit: For some reason I'm suddenly getting this:

dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch upstream
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch origin
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git checkout master
error: pathspec 'master' did not match any file(s) known to git.

This wasn't an issue earlier today, and now I can't rebase off of master with this. Ideas?

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 21e5f23 to 4ddc149 Sep 4, 2018

@graingert

This comment has been minimized.

Copy link
Collaborator

commented Sep 5, 2018

@dargueta I think you want git pull --rebase=interactive upstream master

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from 4ddc149 to fca5f44 Sep 27, 2018

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Sep 27, 2018

Okay, cleaned them up now.

tests/test_copy_command.py Outdated Show resolved Hide resolved

@dargueta dargueta force-pushed the dargueta:copy-parquet branch from fca5f44 to d72cddb Sep 28, 2018

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Oct 19, 2018

I think I've got everything fixed. Any other thoughts?

@dargueta

This comment has been minimized.

Copy link
Contributor Author

commented Nov 27, 2018

*bump*

@jklukas
Copy link
Collaborator

left a comment

lgtm!

@jklukas jklukas merged commit 9fc8d93 into sqlalchemy-redshift:master Nov 29, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@dargueta dargueta deleted the dargueta:copy-parquet branch Nov 29, 2018

@jklukas

This comment has been minimized.

Copy link
Collaborator

commented Dec 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.