Election: Don't send election messages without quorum

[baumanj]

In elections we have the concept of quorum as defined by the butterfly server's check_quorum method. Functionally, it means that enough of the supervisor nodes we know about are healthy such that we can have an election without fear of getting into a split-brain scenario.

However, because quorum is based on the membership protocol and the membership protocol is subject to seeing other nodes as unhealthy due to network congestion, it can be hard to complete an election when the network is congested. Unfortunately adding the election messages into the mix makes congestion worse.

Currently, an election can't finish without quorum, but otherwise the lack of quorum has no effect on how elections run. To avoid unnecessary work that may worsen congestion, we should investigate the possibility of only initiating (and possibly propagating) election messages when there is quorum.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.