Enable replicaSet support for MongoDb by twsouthwick · Pull Request #5712 · microsoft/aspire

twsouthwick · 2024-09-14T01:50:06Z

Description

This change adds support for replica sets in MongoDb. In order to support this a couple changes are required:

Connection strings have query parameters as options that define the replica set. To support this, the expression builder is now called manually to build it up, as well as changing the database connection string to no long reference the server connection string directly (as the server connection string may have the replicaSet option)
The target port and the exposed port must be the same for replica sets to work. This adds a check that updates the MongoDb server to not be proxied and have the same port. Ideally, if no port is set, we should use a random one but I couldn't figure out how to do that in an idiomatic way.

Fixes #5238

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No
Does the change require an update in our Aspire docs?
- Yes
  - Link to aspire-docs issue:
- No

Microsoft Reviewers: Open in CodeFlow

davidfowl · 2024-09-16T21:22:26Z

+    {
+        if (builder.Resource.TryGetLastAnnotation<MongoDbReplicaSetAnnotation>(out _))
+        {
+            throw new InvalidOperationException("A replica set has already been added to the MongoDB server resource.");


Why throw instead of noop? Because of the name?

yeah. I can check if there's already one with the same name and return in those cases if you want

twsouthwick · 2024-09-16T21:30:40Z

+            var dir = Path.Combine(Path.GetTempPath(), "aspire.mongo", Path.GetRandomFileName());
+            Directory.CreateDirectory(dir);
+
+            var rsInitContents = $$"""rs.initiate({ _id:'{{replicaSet}}', members:[{_id:0,host:'localhost:{{port}}'}]})""";


@davidfowl I'd prefer to be able to just docker exec into the container after it started, but couldn't figure out how to do that so I'm running a container and configuring things to ensure it completes before anything else needs to use the db. If there is a better way to do this with aspire things, let me know

We're adding support for this in Aspire 9 very soon. Might be better to wait until that support is available in the app model.

cool - is there a tracking issue for that?

@davidfowl any progress on being able to exec into the container?

eerhardt · 2024-09-17T15:08:55Z

+        // See the conversation about setting up replica sets in Docker here: https://github.com/docker-library/mongo/issues/246
+        static string GetReplicaSetInitDockerfileDir(string replicaSet, string host, int port)
+        {
+            var dir = Directory.CreateTempSubdirectory("aspire.mongo").FullName;


A couple notes here:

Who cleans up this directory?

On Unix, this is going to create a directory that only the current user can access. If the container is running on non-root, the container won't be able to access this folder.

is there a pattern that already exists to handle creating/cleaning up directories? Maybe another reason as @davidfowl mentioned above to maybe wait to exec into a container

So after some thought, I think a pattern we can use here is:

Put the docker file in the nuget package

Mark it as content so it gets copied to the output (we'lll want to name it so it doesn't conflict)

Use build arguments to parameterize the docker file and pass that state in

With #7136, we have now introduced a new "IAspireStore" API:

https://github.com/dotnet/aspire/blob/acdf5012407c2ab0259f4b6f075acb6c668949f3/src/Aspire.Hosting/ApplicationModel/IAspireStore.cs#L17-L22

This file could be written there and my above concerns would be resolved.

twsouthwick · 2024-09-18T19:41:54Z

/azp run

azure-pipelines · 2024-09-18T19:42:00Z

Commenter does not have sufficient privileges for PR 5712 in repo dotnet/aspire

mitchdenny · 2024-09-19T07:48:44Z

                                                        .WithHttpEndpoint(targetPort: 8081, name: "http")
-                                                        .ExcludeFromManifest();
+                                                        .ExcludeFromManifest()
+                                                        .WaitFor(builder);


I'm not sure about using WaitFor invisibly like this. I can see the benefit but if one of the mongo db instances files to start then you won't be able to inspect any of the other databases.

I added that because once you have a replica set, the existing connection string may cause express to fail if it tries to connect before the replica set is initialized (a race condition) and it doesn't recover from it.

creyke · 2024-10-28T16:44:43Z

+1 for this. Missing replica sets doesn't just block transactions, it also blocks change streams.

These are two very commonly utilised features.

This looks close to done, but is not active... is this likely to get merged @twsouthwick?

Thanks in advance.

twsouthwick · 2024-10-28T16:57:00Z

@creyke I'd like to get this in, but in order to set up the replica set, we must run some commands in the container. sounds like docker exec support will be available soon, so this is waiting on that. Once that is available, I'll update it so this can be merged.

feritzcan2 · 2024-12-19T21:25:52Z

@creyke I'd like to get this in, but in order to set up the replica set, we must run some commands in the container. sounds like docker exec support will be available soon, so this is waiting on that. Once that is available, I'll update it so this can be merged.

Still no luck? Thats highly blocker for integration testing /(

alexjamesbrown · 2025-01-13T13:43:05Z

@twsouthwick was there anything left to do on this? Currently using the workaround on this issue
#6811 - but definitely feels like this support should be in Aspire core.

davidfowl · 2025-01-15T08:42:19Z

    private static void ConfigureMongoExpressContainer(EnvironmentCallbackContext context, MongoDBServerResource resource)
    {
-        // Mongo Exporess assumes Mongo is being accessed over a default Aspire container network and hardcodes the resource address
+        var sb = new StringBuilder($"mongodb://{resource.Name}:{resource.PrimaryEndpoint.TargetPort}/?directConnection=true");


ReferenceExpressionBuilder

danmoseley · 2025-02-02T20:35:10Z

@twsouthwick is this still in progress?

davidfowl · 2025-02-02T23:57:31Z

@twsouthwick @eerhardt and I have an offline thread about this one.

alexjamesbrown · 2025-02-17T10:05:07Z

Just commenting again as not to close this issue

@davidfowl / @eerhardt was there any update on this at all?

eerhardt · 2025-02-18T15:41:36Z

+    ///        .WaitFor(messaging);
+    /// </code>
+    /// </example>
+    public static IResourceBuilder<T> WaitFor<T>(this IResourceBuilder<T> builder, IResourceBuilder<IResource> dependency, bool includeHealthChecks) where T : IResource


I wonder if bool includeHealthChecks should instead be a WaitType. We currently have WaitUntilHealthy and WaitForCompletion. But could add WaitUntilStarted.

twsouthwick · 2025-03-12T22:16:56Z

I took a stab at updating it, but have found that since I did the first attempt that, password auth has been added. That adds a complication for enabling replica sets as that then requires a certificate to be used as well.

The place I'm getting stuck on is that I can generate a cert, but getting it into the container is weird.

If I do it as one of the mongo init scripts, I can generate it, but then I can't set the permissions correctly
I've seen people attempt to do it by doing it in the entrypoint (i.e. [some commands to create cert and set permissions]; call_original_entrypoint.sh $@) but it appears the Aspire doesn't allow setting an entrypoint to an arbitrary command like that.

danmoseley · 2025-05-01T20:42:11Z

next action here seems to be a question for @eerhardt and I know he's pretty engaged right now.

eerhardt · 2025-05-01T21:52:58Z

@twsouthwick @davidfowl and I met offline and had some ideas how to move this forward. We think if we had something like #6481, we could make this simpler.

danmoseley · 2025-05-08T15:04:28Z

OK I'm going to close this as it sounds like we're blocked on that. Feel free to reopen, or create a new PR, when ready to continue @twsouthwick

ghost added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Sep 14, 2024

dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Sep 14, 2024

twsouthwick force-pushed the mongodb-replica branch from c5a1982 to ff89d50 Compare September 14, 2024 02:08

Enable replicaSet support for MongoDb

350ce9f

twsouthwick force-pushed the mongodb-replica branch from ff89d50 to 350ce9f Compare September 14, 2024 02:12

build-analysis Bot mentioned this pull request Sep 14, 2024

Flaky test - QdrantFunctionalTests.WithDataShouldPersistStateBetweenUsages #5140

Closed

2 tasks

twsouthwick marked this pull request as draft September 14, 2024 03:56

davidfowl added mongodb Issues related to mongodb integrations area-integrations Issues pertaining to Aspire Integrations packages and removed area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication labels Sep 14, 2024

jakoss mentioned this pull request Sep 14, 2024

MongoDB Transactions requires replicas #5238

Open

handle custom port

3b483d8

davidfowl reviewed Sep 16, 2024

View reviewed changes

Comment thread src/Aspire.Hosting.MongoDB/MongoDBBuilderExtensions.cs Outdated

davidfowl reviewed Sep 16, 2024

View reviewed changes

Comment thread src/Aspire.Hosting.MongoDB/MongoDBBuilderExtensions.cs Outdated

twsouthwick commented Sep 16, 2024

View reviewed changes

twsouthwick added 3 commits September 16, 2024 14:35

per feedback

70e1b29

remove extra usings

a3c4fd2

Merge remote-tracking branch 'upstream/main' into mongodb-replica

f34d9bb

davidfowl reviewed Sep 17, 2024

View reviewed changes

Comment thread src/Aspire.Hosting.MongoDB/MongoDBServerResource.cs

eerhardt reviewed Sep 17, 2024

View reviewed changes

Comment thread src/Aspire.Hosting/ResourceBuilderExtensions.cs

eerhardt reviewed Sep 17, 2024

View reviewed changes

Comment thread src/Aspire.Hosting.MongoDB/MongoDBDatabaseResource.cs Outdated

twsouthwick added 2 commits September 17, 2024 09:52

use referenceexpressionbuilder

727264e

add healthcheck tests

6d4ac1d

build-analysis Bot mentioned this pull request Sep 17, 2024

Some tests timing out on helix #5247

Closed

twsouthwick marked this pull request as ready for review September 18, 2024 19:42

twsouthwick requested a review from mitchdenny as a code owner September 18, 2024 19:42

mitchdenny reviewed Sep 19, 2024

View reviewed changes

davidfowl reviewed Jan 15, 2025

View reviewed changes

danmoseley added the needs-author-action An issue or pull request that requires more info or actions from the author. label Feb 2, 2025

dotnet-policy-service Bot added the no-recent-activity label Feb 17, 2025

davidfowl removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Feb 17, 2025

dotnet-policy-service Bot removed the no-recent-activity label Feb 17, 2025

eerhardt reviewed Feb 18, 2025

View reviewed changes

danmoseley closed this May 8, 2025

github-actions Bot locked and limited conversation to collaborators Jun 8, 2025

Conversation

twsouthwick commented Sep 14, 2024 • edited by dotnet-policy-service Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Microsoft Reviewers: Open in CodeFlow

Uh oh!

davidfowl Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

twsouthwick Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

twsouthwick commented Sep 18, 2024

Uh oh!

azure-pipelines Bot commented Sep 18, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

creyke commented Oct 28, 2024

Uh oh!

twsouthwick commented Oct 28, 2024

Uh oh!

feritzcan2 commented Dec 19, 2024

Uh oh!

alexjamesbrown commented Jan 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danmoseley commented Feb 2, 2025

Uh oh!

davidfowl commented Feb 2, 2025

Uh oh!

alexjamesbrown commented Feb 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

twsouthwick commented Mar 12, 2025

Uh oh!

danmoseley commented May 1, 2025

Uh oh!

eerhardt commented May 1, 2025

Uh oh!

danmoseley commented May 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

twsouthwick commented Sep 14, 2024 •

edited by dotnet-policy-service Bot

Loading

davidfowl Sep 16, 2024 •

edited

Loading

twsouthwick Sep 16, 2024 •

edited

Loading