Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding DMS serverless options to aws provider #39954

Open
2 tasks done
prakharcode opened this issue May 30, 2024 · 4 comments
Open
2 tasks done

Adding DMS serverless options to aws provider #39954

prakharcode opened this issue May 30, 2024 · 4 comments
Assignees
Labels

Comments

@prakharcode
Copy link
Contributor

Description

Sometime back AWS launched serverless DMS (Data Migration Service). This lets users create replication jobs without explicitly setting up a replication instance.

Currently, in the AWS provider, we have support for classic DMS, which requires users to set up a replication instance. It would be nice, if we can provide dms serverless support using the following boto3 functions:

Use case/motivation

Giving airflow[aws] a serverless DMS option would unlock specific DMS use cases that are becoming increasingly common via serverless, like -- full-load of a new table and patching a CDC table using the full load on serverless.

The serverless implementation is also quite appealing for source database replication that happens at a larger frequency than a couple of days (where a dedicated instance doesn't make sense) or with source data that changes or grows a lot.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@prakharcode prakharcode added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet labels May 30, 2024
Copy link

boring-cyborg bot commented May 30, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@Taragolis Taragolis added provider:amazon-aws AWS/Amazon - related issues area:providers and removed needs-triage label for new issues that we didn't triage yet labels May 31, 2024
@Taragolis
Copy link
Contributor

Feel free to raise a PR.

@ellisms
Copy link
Contributor

ellisms commented Sep 25, 2024

If there hasn't been any work on this, I can pick it up.

@eladkal
Copy link
Contributor

eladkal commented Oct 4, 2024

If there hasn't been any work on this, I can pick it up.

Assigned

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants