Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add COPY command support for Parquet and ORC #150

Merged
merged 5 commits into from Nov 29, 2018

Conversation

@dargueta
Copy link
Contributor

@dargueta dargueta commented Aug 3, 2018

Closes #151

  • Allow choosing Parquet and ORC as load formats (see here).
  • Allow choosing fixed_width as a load format as well for consistency with the others. Enforce the presence of the field widths argument if Formats.fixed_width is selected.

Todos

  • MIT compatible
  • Tests
  • Documentation
  • Updated CHANGES.rst
@dargueta dargueta force-pushed the copy-parquet branch 2 times, most recently from 0c3e29e to 9c62cdf Aug 4, 2018
@dargueta dargueta changed the title Add COPY command support for Parquet and ORC Add COPY command support for Parquet, ORC, and Snappy Aug 4, 2018
@dargueta dargueta changed the title Add COPY command support for Parquet, ORC, and Snappy Add COPY command support for Parquet and ORC Aug 4, 2018
@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Sep 4, 2018

@graingert or @jklukas you look like the main contributors here. Sorry to be a pain but I just wanted to make sure someone saw this. Thanks!

Loading

@graingert
Copy link
Member

@graingert graingert commented Sep 4, 2018

Hey there, can you split your Travis/tox changes into another PR, thanks!

Loading

@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Sep 4, 2018

can you split your Travis/tox changes into another PR, thanks!

Done!

Loading

@graingert
Copy link
Member

@graingert graingert commented Sep 4, 2018

LGTM, can you clean up your commits with git rebase?

Loading

@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Sep 4, 2018

There was a bug in one of my tests I had to fix after your LGTM. Do you mean I should squash the commits, or do git rebase origin/master?

Edit: For some reason I'm suddenly getting this:

dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch upstream
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch origin
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git checkout master
error: pathspec 'master' did not match any file(s) known to git.

This wasn't an issue earlier today, and now I can't rebase off of master with this. Ideas?

Loading

@graingert
Copy link
Member

@graingert graingert commented Sep 5, 2018

@dargueta I think you want git pull --rebase=interactive upstream master

Loading

@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Sep 27, 2018

Okay, cleaned them up now.

Loading

tests/test_copy_command.py Outdated Show resolved Hide resolved
Loading
@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Oct 19, 2018

I think I've got everything fixed. Any other thoughts?

Loading

@dargueta
Copy link
Contributor Author

@dargueta dargueta commented Nov 27, 2018

*bump*

Loading

Copy link
Member

@jklukas jklukas left a comment

lgtm!

Loading

@jklukas jklukas merged commit 9fc8d93 into sqlalchemy-redshift:master Nov 29, 2018
1 check passed
Loading
@dargueta dargueta deleted the copy-parquet branch Nov 29, 2018
@jklukas
Copy link
Member

@jklukas jklukas commented Dec 11, 2018

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

3 participants