Clustering

[Svedrin]

Adding a cluster feature would be nice. We could:

• Correctly alert on a node being down (as opposed to multiple separate alerts that pings are dropping)
• When that happens, use a surviving node to view the dead node's data up until it died (see also Draw Histograms using peers' data #19 )
• Sync targets in a much nicer way (the current implementation handles target deletions and renames pretty badly)

Rough idea for an implementation:

• We'll probably need some kind of Raft implementation, so that nodes can elect a leader (or at least know if they're in Quorum)
• While nodes are out of quorum, they should continue to ping their targets and collect data. They should not send out alerts or try to send their data to other nodes though.
• It would probably be awesome if it didn't matter which other node a node talks to, as long as that other node is in quorum. This would make it easier to use MeshPing in a network where not all the nodes can see each other.

Tech ideas:

• Nodes will have to distribute information to other nodes. Something like ZeroMQ would probably make this tremendously easier because it supports PubSub. So, a node could connect to another node, and if that node is in quorum, subscribe to its updates and get all the cluster's data. ZMQ could run on port 9923.
• Join should work via the HTTP API. Accepting joins should probably be the master's responsibility.
• For authentication, nodes should sign their messages using libsodium. Unsigned messages should be discarded.
• Local targets should be in a separate database table. This makes them easier to distinguish from cluster targets and probably also makes it easier to operate a single node.
• It would be awesome if nodes didn't even have to be a Meshping instance, but could also be an ESP or Arduino that just pulls a list of targets from an actual Meshping instance and delivers ping data back. Probably not all that helpful in a professional setting, but could be interesting for home users or people monitoring a distributed WiFi network.