[Enhancement]: support for documentdb global cluster failover/recovery

[sarahbhojwani]

Description

Use Cases

Having DocumentDB as a global cluster is awesome, but only useful if we can perform failover/recovery during an incident.

Failover Process:

• remove secondary cluster from global (promotes to standalone regional cluster)

Recovery Process:

• Delete old global and primary cluster
• Convert newly-promoted regional cluster to a global cluster
• Add regional cluster to the global as a secondary cluster in the primary region (don’t forget to add instances)
• Wait for data replication to complete (db status becomes available)
  …at this point the secondary region is still listed as the primary cluster. To switch back, implement the full failover process again.
• Remove secondary from global
• Delete old global and primary
• Convert newly-promoted regional cluster to global
• Add regional cluster to global as a secondary cluster in secondary region (don’t forget instances)

Attempted Solutions

This is currently done with Lambdas and DynamoDB table from AWS side, which is a lot of overhead. The same steps can also be done through CLI commands, with the only blocker being on waiting times (replication to complete, db becomes available before trying to add instances).

Proposal

Create failover processes that are natively managed in terraform. Even if it is just the option to maintain state while removing clusters from the global or converting to global/adding regions to global.

Affected Resource(s) and/or Data Source(s)

aws_docdb_global_cluster

Potential Terraform Configuration

No response

References

https://docs.aws.amazon.com/documentdb/latest/developerguide/failover.html

https://catalog.us-east-1.prod.workshops.aws/workshops/464d6c17-9faa-4fef-ac9f-dd49610174d3/en-US/globalcluster/failover

https://github.com/awslabs/amazon-documentdb-tools

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[indrabasak]

@justinretzolk What's the status of this story? Any updates?

[justinretzolk]

Hey @indrabasak 👋 Thank you for checking in on this! Unfortunately I can't provide an ETA on when this will be looked into due to the potential of shifting priorities. We prioritize by count of 👍 reactions and a few other things (more information on our prioritization guide if you're interested).

[pedro-geraghty-sw]

@justinretzolk What's the status of this story? Any updates?

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!