AWS DocumentDB support with MongoDB container

[Nikhiladiga]

I was reviewing the services offered in LocalStack and came across their DocumentDB implementation:
https://docs.localstack.cloud/aws/services/docdb/

It looks like they’re running a MongoDB-compatible server under the hood, and they’ve also documented the current limitations clearly.

Given this approach, I think we could implement something similar to how RDS is handled.
@Nahuel990 — what do you think?

[Nahuel990]

Hey @Nikhiladiga I’m not against DocumentDB... as you've already seen the implementation would follow the same pattern as RDS (spin up a real MongoDB container via the Docker socket and return the endpoint). We already do this for MySQL, Redis, etc

Our current priority is expanding coverage for services that users are more frequently blocked on like CloudFormation provisioners (we have 68 resource types but there are still gaps), Cognito federation flows, and Terraform compatibility fixes (the new TF is OpenTofu and several companies are migrating)

If you’re looking to contribute, here are some higher-impact areas:

CloudFormation (CFN): missing provisioners for AWS::ElastiCache::CacheCluster, AWS::RDS::DBInstance, and ECS service updates
ResourceGroupsTagging API completely missing;

DocumentDB is a good idea for the roadmap but nobody asked for it yet, though. If none of the above interests you and you’d prefer to work on DocumentDB, feel free to open a PR (be sure we don't infringe MongoDB license as they are not fully OpenSource)

[Nikhiladiga]

Makes sense. I'll check out the missing pieces mentioned above before diving into this.

[Nahuel990]

We also have a ticket open for EKS which could follow the same pattern (it's in the issues section) There's no need to implement it in one go, as it could be done in separate parts

[xPTM1219]

To add to this discussion, I have started implementing DocumentDB service. It is still a work in progress and it does not work currently, there is a lot to do. The code mainly reflects RDS because I copied most of the implementation but I started going through it to document and gear the code towards DocDB.

Of course, any contribution and/or comments are welcome. Also, I have not opened a Draft pull request yet because it does not run, my plan is to get something basic running and then I'll open a pull request.

The current work can be found in my repo.

Also, I am using this page Amazon DocumentDB (with MongoDB compatibility) where AWS details what it's supported in terms of aws cli.

[Nahuel990]

@xPTM1219 before landing this I'd like to understand better what is the use case and if you've checked the Contributing guidelines. We had a few cases from users implementing services from scratch with knowledge about how the service work and that takes more time on our side reviewing the code rather than implementing it. We are not against adding new features/services as long as you are an avid user of it and have a clear understanding/use case for it

[xPTM1219]

Hi @Nahuel990, I have read the contributing guidelines on the repo and is precisely why I have not opened any pull request, because is not ready nor complaint. I posted the update here just for general knowledge and to avoid anyone stepping in each other foot.

I want to start with basic functions to establish the basic code for DocDB:

• Creating a cluster and instance
• Deleting a cluster and instance
• Stopping a cluster and instance
• Describing the clusters and instances
• Maybe tagging

Note that eventually the goal is to get this list implemented.

My idea is to make use of Mongo containers when managing instances from aws-cli instead of handling every Mongo command. Now, I am aware that DocDB does not support all Mongo operations (detailed here), threfore, I have no plans to handle the Mongo side of operations, I want to simply create a bridge between AWS DocDB and MongoDB, but the code will include the reference to the page so that any user that is using DocDB in Ministack is aware of what to implement or not.

In terms of usage, the reason I am working to get this added is because my team at work and I will be using it to test our CDK code with it. DocumentDB is expensive and we don't want to be spending money just to "test". Starting today, one coworker started working developing the CDK code for DocDB and I am in charge of transitioning our 3 different Mongo servers in 3 different environments(Prod, Test and Develop) to DocumentDB. Having a tool like Ministack is definitely helpful.

Also, today I developed a demo of S3 webpage hosting in Ministack and handed that demo to one of the coworker working with a new UI we are going to develop. So big thank you for Ministack!

[xPTM1219]

For historical purposes, I have added the DocumentDB service in the pull request DocumentDB addition #1715.

The pull request was closed without a specific reason about the guidelines, so I could not fix it, but at least the code is there if you need it @Nikhiladiga . I think it is because I forgot to create an issue.

I have a repo in my Github that tests CDK against Ministack with DocDB, the link is below. I have two versions, Python and Java. I also have code that tests the database but those connect to the Mongo container so you can ignore them.

• My repo CDK test against Ministack with DocDB
• My repo DocDB addition

[Nahuel990]

We started requesting to open an issue first around 4 months ago as many people wanted to add full parity of AWS services (and also we were receiving a lot of PRs coming from AI with people not understanding the structure of the project)

https://github.com/ministackorg/ministack/blob/main/CONTRIBUTING.md#for-new-services--open-an-issue-first

About the DocumentDB PR I haven't looked at it in depth but I also saw you are attaching a MongoDB container to it... AWS provides 2 separate services MongoDB and DocumentDB, DocumentDB is not 100% compatible with MongoDB (there is a whole page stating diff per each version), it's not like RDS where the engine actually runs on MySQL or PostgreSQL

DocumentDB also separates the storage from the database workload so it's a different structure

[xPTM1219]

Hi @Nahuel990 thanks for the reply.

I apologize on completely forgetting to open the issue, that is my bad. I am currently writing the contents for the issue.

My goal is to start supporting the side of MongoDB support, it is a good addition for the future to add DocumentDB in it's entirety.

[Nahuel990]

@xPTM1219 I was re-reviewing your closed PR. We can reopen it, but the data plane must use the DocumentDB container instead of MongoDB (https://github.com/documentdb/documentdb Image: ghcr.io/documentdb/documentdb/documentdb-local:latest (amd64 and arm64) It's MIT licensed, so there's no MongoDB license concern, and it's actively maintained with AWS on its steering committee and It speaks the same wire protocol, and the AWS parity (control plane, errors, supported features) should be implemented on our side (most of your PR already has it)

[xPTM1219]

@Nahuel990 Awesome! I am going to work on that, thank you for pointing that out. Also, I will open an issue to follow the Contributing guidelines and reference this discussion.
Issue here - #1841