Balanced peers in kademlia

[zbiljic]

relates to: #57

Using base address for starting node of 00000000 rest of description is made here.
For that base address we have following (random) addresses for different proximities (first 8 here):

0 -> "11011011"
1 -> "01011110"
2 -> "00100010"
3 -> "00010111"
4 -> "00001000"
5 -> "00000100"
6 -> "00000010"
7 -> "00000001"

Bits in common are what defines bin of some other address in relation to base address.

To have some sort of balancing, we try to have some dispersion of addresses for each bin (proximity).
We define this as bit suffix for bin, and if should be number > 0. If we set it to 1 we would have 2 possibilities, with 2 we have 4, with 3 we have 8, and so on.

For 2 following suffixes are produced for address:

00
01
10
11

Using this config, for each bin we generate table of "balanced" addresses toward which we aspire to have connections to (this is generated on Kademlia instantiation and is based on node 'base' address). We use prefix for each bin as a staring point, and to it we append each of these suffixes, to create pseudo address prefixes:

bin(0)
prefix = ""
balanced prefixes:
'100'
'101'
'110'
'111'

bin(1)
prefix = "0"
balanced prefixes:
'0100'
'0101'
'0110'
'0111'

bin(2)
prefix = "00"
balanced prefixes:
'00100'
'00101'
'00110'
'00111'

...

bin(4)
prefix = "0000"
balanced prefixes:
'0000100'
'0000101'
'0000110'
'0000111'

...

The additional bit in these examples comes after the prefix, and is opposite of the bit of the one in the base address. Index of that bit is at PO + 1.

With these prepared "balanced" address prefixes, for each bin we can check if we have balanced connections. We also use this information to decide if we want to connect to some other known peer in order to have balanced connected state.

The algorithm to determine if we need to connect to some other peer is done by:

• going over each generated pseudo address (prefix) from each bin
• then finding closest already connected peer to that pseudo address
• if the proximity of that closest connected peer (to pseudo address) is less than 'bin + suffix length'
• we connect to the closest known peer
  • closest known peer is again found in relation to pseudo address
  • and if proximity of it in relation to pseudo address is what we expect for balanced connection we attempt to connect to it

This process is added as separate function just before existing process for connecting to peers in Kademlia, which looks at neighborhood depth to decide if more peer connections need to be made.

[zbiljic]

/run beekeeper

[zbiljic]

/run beekeeper

[acud]

Looks good. Let's merge this after the release tomorrow

[metacertain]

some occurences of a problem with check lengths, probably we should add some simple test as well that could show this

[metacertain]

I think we need i + k.bitSuffixLength + 1 here because of the extra flipped bit in the common address
Also in line 284

[metacertain]

I think this should be < i+k.bitSuffixLength+1

[metacertain]

This could be substituted with checking proximity of closest connected peer

[metacertain]

I think this should be int(bin) + k.bitSuffixLength + 1 because of flipped bit in common addresses

[acud]

@metacertain let's do a revision of the branch before merging it

[metacertain]

So one other thing that was missing, is that actually we need an extended Proximity function that has a higher limit than MaxPO
Because, if we want balance connections in (for example) bin [MaxPO-1], and suffix length is 3, than we need to be able to see proximities beyond MaxPO ( actually MaxPO-1 + suffix length + 1 ). So I duplicated this Proximity function as ExtendedProximity, after being riddled by anomalies beginning at PO 13. It was not obvious at all.

I think we can merge this now!

[metacertain]

/run beekeeper

[metacertain]

/run beekeeper

[ralph-pichler]

/run beekeeper