Dispersy walker has fewer candidates than expected

[boudewijn-tribler]

We would expect more candidates sooner. Especially the AllChannelCommunity takes much longer to obtain a good number of candidates than we would expect.

Strangely enough the walk success rate is relatively high (around 80%). This contradicts the lack of available candidates.

This behaviour must be either solved or explained. Please investigate.

[boudewijn-tribler]

Option: perhaps the majority of the peers that we contact do not return new peers for some reason? This would explain why we get few candidates while the walk success rate is relatively high. This should be fairly easy to confirm of deny.

[synctext]

Reproduced this on my Ubuntu machine.
Instead of expected 21 candidates I'm seeing as low as 4 candidates.
This is with a fully connectable Tribler instance.

[synctext]

After 15 minutes there are 0 candidates in Search community. Search community is broken in current devel.
Strangely enough, AllChannels is bursting with peers, sometimes 27 peers.

From git://github.com/Tribler/tribler
e34d4e2..5fe13d3 devel -> origin/devel
Updating e34d4e2..5fe13d3

[synctext]

The mystery continues to unfold.

Load balancing needs to be improved, both bootstrap_candidates and normal peers. There is a bit of hammering:

• 2 bootstrap_candidates get: 109 + 110 = 219 connections
• 10 bootstrap_candidates get: 52 connections in total

Some normal peers also are hammered, see below.

[synctext]

More details on how walks fail after running for almost 15hours:

[boudewijn-tribler]

Ended up being a bug in the tracker. Long story short, the tracker's response messages would use the highest 'community version number' that the tracker knew of. Hence, once someone talked to the trackers using version 0002 it would respond to any subsequent request (from any other peer in that community) with version 0002. This response would then be dropped because this version is unknown.

To fix this the tracker code has been modified to only respond with version 0000. All trackers are still running the older svn based code and we will not update svn. Instead, we will fix the code on git and slowly migrate our trackers to the latest git.