WIP: Added latency community

[qstokkink]

This PR adds the (highly experimental) latency community. The latency community is not enabled by default. To work with the latency community:

1. Create a new Network class (very important) for the LatencyCommunity.
2. Load the LatencyCommunity with only the LatencyEdgeWalk strategy into IPv8.
3. Fetch your partners from the community.accepted_proposals set.

LatencyEdgeWalk

The LatencyEdgeWalk strategy will collect a set number of peers from the bootstrap nodes/trackers, 30 by default. Next, the strategy will attempt to get successive network-unique ping-unique (0.05 seconds similarity) peers from each of these 30 edge roots, up until a given edge length (6 by default). As the edges are being grown, the strategy also provides ping measurements if needed.

After each step in the discovery process the LatencyEdgeWalk strategy will update its registered overlay by setting the overlay's possible_peers attribute to the current list of peers included in the disconnected graph of edges (the ancestry datastructure).

Every so often (by default 30 seconds), the strategy performs a cleanup of all open connections to peers which are not included in the ancestry.

At the end of each cycle we distinguish two types of peers:

• Open connections which cannot be included in a latency-unique edge, we will dub this the total peers.
• Open connections which are part of a latency-unique edge, which we dub the possible peers.

LatencyCommunity

The LatencyCommunity is in charge of the matching of peers. The Community predominantly uses the PeerSelector to advise any matching with others. The overlay itself periodically (by default every 5 seconds) (a) sends out proposals and keeps a window (30 by default) of optimal matches or (b) breaks the current worst match it has if the amount of matches is equal to the required amount (60 by default). Asynchronously the LatencyCommunity may decide to accept or reject incoming proposals.

The LatencyCommunity provides the last class of peers:

• Peers which have bilaterraly agreed to form a matching, we dub these matched peers.

PeerSelector

This class uses kerning to decide the error of including a peer into a given existing set of choices, according to a specified target distribution. The idea is that a programmer gives both (1) a continuous function (lambda x: preferred_count / x by default) and (2) a list of x-coordinates (bins) to evaluate the function (0.001 through 2.001 with steps of 0.05 by default). This function need to be scaled precisely for the kerning, this is handled automatically by generate_reference for a given preferred peer count and the current binning distribution. We use Gaussian kernel density estimation.

Gumby & Jenkins

My personal Jenkins experiment for this PR can be found here: https://jenkins-ci.tribler.org/job/pers/job/latency_noodle_qstokkink/. The source code for the experiment can be found here: https://github.com/qstokkink/gumby/tree/latency_overlay/experiments/latency_overlay.

Future work

Even though the basic concept works, the CPU utilization should be improved and the matching algorithm will require some further love.

[qstokkink]

I'm going to give up on finishing it today. I cannot get the matching algorithm to behave reliably with differing peer latency distributions.

[qstokkink]

Ready for review, let me know if the instructions in the O.P. require supplemental info.

[qstokkink]

Marking this as W.I.P. to fix the matching issues within this PR.

[qstokkink]

Probably not a good idea to merge this, closing for now.