-
Notifications
You must be signed in to change notification settings - Fork 3
To Do
A list of things to do.
Currently, each node determines failure of other members independently. There are several nice, scalable consensus protocols for gossip that could easily be incorporated. Incorporating consensus would provide very strong group membership semantics.
Right now it's kind of slid under the rug in the initial gossip - which is quite ugly, and the reply - which is okay. Really needs to be its own message, triggered off of an empty digest list when discovering.
The current scheme works well for moderately sized groups, but can get expensive quickly. All the cool kids create hierarchical structures based on the IP subnet. This allows inexpensive gossip within the subnet and the correspondingly quick information dissemination and failure detection there. Between subnets, gossip is less frequent. I have some nice papers outlining this strategy and seems like it will be easy to modify the current code base to accommodate this.
The consensus protocols really like broadcast and when you combine the hierarchical gossip with consensus it turns out that you can now make use of UDP broadcast within a subnet. This opens up some rather nice properties for quicker discovery and important state synchronizing.
Right now, nodes are known by their endpoints. Unfortunately, the consensus protocols would really like to use integer labels for the nodes so that suspicion matrices don't take inordinate amounts of bandwidth to propagate. So you either pre-label nodes with their ID, which is not easy in the large, or....? Handing out IDs - and more importantly, garbage collecting the ID space so it doesn't become sparse (and therefore useless) is extremely difficult to do without some centralized accounting.