ConnectionMultiplexer doesn't always reconnect - No connection available

[JonCole]

We have seen multiple customers who are getting "No connection available" type errors after a network blip and the error persists until the multiplexer is disposed and re-created.

We have verified that the Redis instance is up and responding to other clients fine. In some casese, we have even verified that a different multiplexer in the same application is able to talk to Redis, so this is not a networking or Redis server problem.

Many of our customers set abortConnect=false and expect the multiplexer to always reconnect eventually. Such customers don't have logic to create a new multiplexer and have to reboot their client applications in order to get them back into a working state.

We are continuing to investigate and, if we figure out the bug, we will submit a pull request with the fix, but I wanted to create an issue to track this in case others are seeing this same behavior.

[mgravell]

Vexingly, intemittent network blips (and the resultant carnage) are a nightmare to debug. If you find anything solid, we'll be very happy to look at a PR. I can try to take a look too, but I need to schedule against other tasks.

…

On 25 Jan 2017 5:58 p.m., "Jon Cole" ***@***.***> wrote: We have seen multiple customers who are getting "No connection available" type errors after a network blip and the error persists until the multiplexer is disposed and re-created. We have verified that the Redis instance is up and responding to other clients fine. In some casese, we have even verified that a *different multiplexer in the same application* is able to talk to Redis, so this is not a networking or Redis server problem. Many of our customers set abortConnect=false and expect the multiplexer to always reconnect eventually. Such customers don't have logic to create a new multiplexer and have to reboot their client applications in order to get them back into a working state. We are continuing to investigate and, if we figure out the bug, we will submit a pull request with the fix, but I wanted to create an issue to track this in case others are seeing this same behavior. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#559>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABDsMZx99xR3UvyfiYB3U9n6GSLBQsTks5rV41cgaJpZM4LtzZH> .

[NickoBe]

In case it if of any value I can share that we regularly encounter something very similar effect to this as well. In our case it is specifically tied to failovers, or other types of similar temporary hickups, in our Redis cluster infrastructure (as operated in AWS ElastiCache). Only reliable way of addressing the 'broken redis connectivity' has been to fully restart the applications.

[NickCraver]

@JonCole I recognized this morning that Microsoft.Extensions.Caching.Redis is depending on 1.2.4. I poked the ASP.NET leads on twitter, but it would be very good if this dependency was upgraded to 1.2.6 or higher as we fixed a critical series of network bugs by fixing a build issue that creeped in during the project.json -> .csproj port.

Is there any way to get that dependency upgraded for the 2.1 release, or is it too late? It would likely solve many customer issues.

[JonCole]

@NickCraver thanks for the heads up - I will have a conversation with the right team internally and see what can be done.

[mgravell]

As a general network / connectivity issue, I'm closing this and rolling it into #871 - note: there is good news!

(edit: fat-fingered the issue number)