Skip to content

/ sqlalchemy-redshift Public

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make TRUNCATECOLUMNS optional, also add all other parameters #52

Merged
merged 3 commits into from Sep 29, 2015
Merged

Make TRUNCATECOLUMNS optional, also add all other parameters #52

merged 3 commits into from Sep 29, 2015

Conversation

@graingert
Copy link
Member

@graingert graingert commented Sep 24, 2015

No description provided.

@graingert graingert force-pushed the add-missing-copy-command-args branch 8 times, most recently Sep 24, 2015
@graingert graingert changed the title add missing copy command args WIP: add missing copy command args Sep 24, 2015
@graingert graingert force-pushed the add-missing-copy-command-args branch 3 times, most recently Sep 24, 2015
@graingert
Copy link
Member Author

@graingert graingert commented Sep 26, 2015

this breaks backwards comparability by removing some defaults eg:

        truncate_columns=True,
        delimiter=',',
        ignore_header=0,
        empty_as_null=True,
        blanks_as_null=True,

@graingert graingert changed the title WIP: add missing copy command args Add missing copy commands, allow disabling TRUNCATECOLUMNS Sep 26, 2015
@graingert graingert force-pushed the add-missing-copy-command-args branch Sep 26, 2015
@graingert graingert changed the title Add missing copy commands, allow disabling TRUNCATECOLUMNS Make TRUNCATECOLUMNS optional, also add all other parameters Sep 26, 2015
@graingert
Copy link
Member Author

@graingert graingert commented Sep 26, 2015

@jklukas I was wondering if I could have a set of AWS credentials that can only upload to an S3 bucket?

(To be able to test this code properly)

@graingert
Copy link
Member Author

@graingert graingert commented Sep 26, 2015

This needs a changelog update, and the docs updating.

@graingert graingert force-pushed the add-missing-copy-command-args branch 4 times, most recently Sep 28, 2015
@graingert
Make TRUNCATECOLUMNS optional, also add all other parameters
bd9b13f
@graingert graingert force-pushed the add-missing-copy-command-args branch to bd9b13f Sep 28, 2015
@graingert
support multiple columns in Copy insert
225f8a0
@graingert graingert force-pushed the add-missing-copy-command-args branch to 225f8a0 Sep 28, 2015
redshift_sqlalchemy/commands.py
# found. After looking at a few session tokens they appear to be the same as
# the aws_secret_access_key pattern, but much longer. An example token can be
# found here:
# http://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html

This comment has been minimized.

Sign in to view
@MichaelAquilina

MichaelAquilina Sep 28, 2015

thats pretty bad tom. http is deprecated
redshift_sqlalchemy/commands.py
# found here:
# http://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html
# The regexs for access keys can be found here:
# http://blogs.aws.amazon.com/security/blog/tag/key+rotation

This comment has been minimized.

Sign in to view
@MichaelAquilina

MichaelAquilina Sep 28, 2015

omg tom
@MichaelAquilina
Copy link

@MichaelAquilina MichaelAquilina commented Sep 28, 2015

Looks good to me 👍

@graingert
upgrade http links to https
1f6077d
@graingert
Copy link
Member Author

@graingert graingert commented Sep 28, 2015

@jklukas @thisfred @bouk @cpcloud: Thoughts?

tests/test_copy_command.py
ACCEPTANYDATE
TIMEFORMAT AS 'auto'
""" % creds
copy = dialect.CopyCommand(

This comment has been minimized.

Sign in to view
@graingert

graingert Sep 28, 2015
Author Member

analogous code appears in the YPlan test suite but hits S3 and Redshift
@graingert
Copy link
Member Author

@graingert graingert commented Sep 29, 2015

@jklukas @thisfred @bouk @cpcloud I'm going to go ahead and merge this and make a release because I need it in production pretty sharpish

graingert added a commit that referenced this pull request Sep 29, 2015
@graingert
Merge pull request #52 from graingert/add-missing-copy-command-args
d341cff 
 Make TRUNCATECOLUMNS optional, also add all other parameters
@graingert graingert merged commit d341cff into master Sep 29, 2015
2 checks passed
2 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@graingert graingert deleted the add-missing-copy-command-args branch Sep 29, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
@graingert @MichaelAquilina