Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Redshift database support #639

Merged
merged 49 commits into from
Aug 26, 2022
Merged

Add Redshift database support #639

merged 49 commits into from
Aug 26, 2022

Conversation

pankajkoti
Copy link
Contributor

@pankajkoti pankajkoti commented Aug 12, 2022

Description

What is the current behavior?

No support available for Redshift database

What is the new behavior?

Add support for Redshift database for the following operators:

  • load_file
  • transform
  • drop_table
  • run_raw_sql
  • append
  • export_file
  • dataframe

Note: The merge operator implementation needs some discussion and will be
provided in a subsequent PR.

Does this introduce a breaking change?

No.

Checklist

  • Created tests which fail without the change (if possible)

closes: #612

@kaxil kaxil added this to the 1.1.0 milestone Aug 24, 2022
@pankajkoti
Copy link
Contributor Author

It would be great if can squash your commit.

Squashing looks difficult as the branch has been rebased multiple times and also few conflicts have been handled with merge commits from main time to time. However, when the branch is merged to main it will appear as a single commit.

Copy link
Contributor

@sunank200 sunank200 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall great work with implementation of adding Redshift @pankajkoti . I have added few comments.

src/astro/databases/aws/redshift.py Outdated Show resolved Hide resolved
.github/workflows/ci.yaml Show resolved Hide resolved
.github/workflows/ci.yaml Show resolved Hide resolved
.github/workflows/ci.yaml Show resolved Hide resolved
example_dags/example_load_file.py Show resolved Hide resolved
src/astro/databases/aws/redshift.py Outdated Show resolved Hide resolved
src/astro/databases/aws/redshift.py Outdated Show resolved Hide resolved
src/astro/databases/aws/redshift.py Show resolved Hide resolved
tests/databases/test_redshift.py Outdated Show resolved Hide resolved
tests/databases/test_redshift.py Show resolved Hide resolved
pankajkoti and others added 3 commits August 25, 2022 15:32
Copy link
Collaborator

@utkarsharma2 utkarsharma2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@sunank200 sunank200 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just deepsource and CI has to run fine

@pankajkoti pankajkoti merged commit d267897 into main Aug 26, 2022
@pankajkoti pankajkoti deleted the 612-add-redshift-support branch August 26, 2022 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Redshift DB - Pandas Path
5 participants