Skip to content

Make TRUNCATECOLUMNS optional, also add all other parameters #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 29, 2015

Conversation

graingert
Copy link
Member

No description provided.

@graingert graingert force-pushed the add-missing-copy-command-args branch 8 times, most recently from e12a380 to d906bef Compare September 24, 2015 17:35
@graingert graingert changed the title add missing copy command args WIP: add missing copy command args Sep 24, 2015
@graingert graingert force-pushed the add-missing-copy-command-args branch 3 times, most recently from 1d48989 to cda4d57 Compare September 26, 2015 00:00
@graingert
Copy link
Member Author

this breaks backwards comparability by removing some defaults eg:

        truncate_columns=True,
        delimiter=',',
        ignore_header=0,
        empty_as_null=True,
        blanks_as_null=True,

@graingert graingert changed the title WIP: add missing copy command args Add missing copy commands, allow disabling TRUNCATECOLUMNS Sep 26, 2015
@graingert graingert force-pushed the add-missing-copy-command-args branch from cda4d57 to bd95ed9 Compare September 26, 2015 00:07
@graingert graingert changed the title Add missing copy commands, allow disabling TRUNCATECOLUMNS Make TRUNCATECOLUMNS optional, also add all other parameters Sep 26, 2015
@graingert
Copy link
Member Author

@jklukas I was wondering if I could have a set of AWS credentials that can only upload to an S3 bucket?

(To be able to test this code properly)

@graingert
Copy link
Member Author

This needs a changelog update, and the docs updating.

@graingert graingert force-pushed the add-missing-copy-command-args branch 4 times, most recently from 8fe388a to b040db5 Compare September 28, 2015 12:58
@graingert graingert force-pushed the add-missing-copy-command-args branch from b040db5 to bd9b13f Compare September 28, 2015 13:26
@graingert graingert force-pushed the add-missing-copy-command-args branch from 47fc319 to 225f8a0 Compare September 28, 2015 16:40
# found. After looking at a few session tokens they appear to be the same as
# the aws_secret_access_key pattern, but much longer. An example token can be
# found here:
# http://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thats pretty bad tom. http is deprecated

@MichaelAquilina
Copy link

Looks good to me 👍

@graingert
Copy link
Member Author

@jklukas @thisfred @bouk @cpcloud: Thoughts?

ACCEPTANYDATE
TIMEFORMAT AS 'auto'
""" % creds
copy = dialect.CopyCommand(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

analogous code appears in the YPlan test suite but hits S3 and Redshift

@graingert
Copy link
Member Author

@jklukas @thisfred @bouk @cpcloud I'm going to go ahead and merge this and make a release because I need it in production pretty sharpish

graingert added a commit that referenced this pull request Sep 29, 2015
 Make TRUNCATECOLUMNS optional, also add all other parameters
@graingert graingert merged commit d341cff into master Sep 29, 2015
@graingert graingert deleted the add-missing-copy-command-args branch September 29, 2015 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants