Log warning when ClusterMembershipOptions.ValidateInitialConnectivity=true #5148
Conversation
…nitialConnectivity=true
| else | ||
| logger.Warn( | ||
| ErrorCode.MembershipSendingPreJoinPing, | ||
| $"${nameof(ClusterMembershipOptions.ValidateInitialConnectivity)} is set to false. This is NOT a supported value for production environment."); |
There was a problem hiding this comment.
I'd say it's supported but not recommended.
|
Just curious, why is that a problem to use this in prod? What kind of issues did you see? |
|
@gabikliot as far as I understand it, the issues come when silos don't have stable identities, which is mostly the case in container hosts or Service Fabric where the port is not fixed. Generally these environments have large churn (rolling upgrades, for ex, or maybe cluster balancing or maintenance). If a silo terminates without updating membership then its successor (with likely a different id) will see it, fail to connect to it, and hence fail to join the cluster until the 5 minute lee-way period is over. @benjaminpetit correct me if I'm wrong - there's probably something wrong in my understanding |
While it can be fine to run with this setting set to true in test environment, it's generally a very bad idea to use it in production.
This PR just add a warning when a silo tries to join a cluster with this setting enabled.