Intermittent network connectivity issues to containers and service-to-service + "Rejoining Server" message in Aspire Dashboard

[jasonshave]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Set up my new workstation and I'm plagued by this 'rejoining server' message. Networking generally seems broken (apps can't connect to my infra containers like Cosmos DB emulator, Azurite, etc.). It doesn't happen all the time. I can restart 2, 3, 4 times and it continues to be an issue, then on the 5th try, it works and is fine for an hour. I can't seem to nail down what's wrong.

Same codebase on another PC works mostly fine including 4 other team member machines. I did get this same issue on my corporate laptop (Lenovo) as well but it was much less often. Sometimes I can get lucky by shutting down Visual Studio, giving it a minute, then starting back up will usually clear things up but not always.

We don't use any proxy servers and there is nothing other than standard Intune enrollment going on.

Here is the Azurite error: System.AggregateException: 'Retry failed after 3 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No connection could be made because the target machine actively refused it. (127.0.0.1:62311))

Here is Redis: StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). Error connecting right now. To allow this multiplexer to continue retrying until it's able to connect, use abortConnect=false in your connection string or AbortOnConnectFail=false; in your code. ConnectTimeout

Running about 25 ASP.NET 8 services and several infrastructure containers like Cosmos DB Emulator, Raven DB, NATS, Azurite.

Expected Behavior

When things work, there are no issues. All services start, containers which are persisted are always available and function properly.

Steps To Reproduce

We have a very large code base so reproducing this would take a lot of work.

Exceptions (if any)

No response

.NET Version info

All services run .NET 8.

Anything else?

Aspire version is 13.2.1
Visual Studio 18.4.3

[jasonshave]

Did some more research on this today within our team and I have at least one other team member who experiences the same thing on their Lenovo P14s (gen5) laptop. He says restarting Visual Studio often solves it for a while.

[jasonshave]

I think we figured out this issue. When we created some of our infrastructure containers we used the 'well known' ports for Raven DB and NATS containers. This meant I had connection strings for raven set at http://localhost:8080 and nats was nats://localhost:4200. Even though the Aspire project had these as accessible ports, I believe the intention is for services and my host PC to always use the published random/dynamic ports Aspire defines. Somehow when accessing these containers using the 'internal' ports, it caused a network shutdown of sorts.

I was able to narrow this down during a crashed state where GitHub Copilot did some basic networking tests and found the smoking gun. After changing the connection strings from 'hard-coded' ports to the dynamic ports obtained by Aspire, the issue went away entirely.

[jasonshave]

Spoke too soon. This is still quite broken. I'll try to resolve and post updates as I find them.