Dispersy: sequence number problem

[synctext]

Details are unknown to me. Had needed a fix in the past.
Still one bug remains. Unit test needed.

[boudewijn-tribler]

From an earlier email conversation:

Concerning the bug with the sequence numbers. There are two problems.

1. A peer made two messages and gave them the wrong global time value.
   This probably results from someone removing their dispersy database
   but not their private key. Hence they start creating messages
   starting at sequence number 1 again.

   This could also be a cause of drops, since there are now two
   messages, both with the same sequence number. Johan's statistics
   picture shows 8866 undo-own message drops by duplicate sequence
   number.

   I vaguely recall that we decided, several years ago, that this
   scenario would not occur for normal users. Perhaps we should revise
   this? If so we can solve this problem by generating a new
   public/private key whenever a user removes the dispersy database. If
   no one objects I will implement this for the new version.

2. A bigger problem is that apparently peers will accept messages that
   have the correct sequence number order but with the wrong global time
   order. This I will fix and I will implement a database check for
   peers that update to a new tribler version that will find and remove
   these invalid messages from the database.

[boudewijn-tribler]

r31912 | boudewijn | 2013-03-11 14:33:13 +0100 (Mon, 11 Mar 2013) | 19 lines

Previously messages that had sequence numbers enabled would not check
the global time ordering. This has resulted in some peers accepting
message A@10#1 and B@9#2. In other words, where the global time
ordering is the reverse of the sequence number ordering.

This problem can occur when a user removes the dispersy.db file but
keeps the files containing the public/private keys. Resulting in a
'reset' of the sequence numbers, and possibly creation of out-of-order
messages.

• bugfix: incoming messages that have sequence numbers enabled (mostly
  undo-own and undo-other messages) will now also check the global
  time ordering, rejecting those that do not match.
• improvement: database upgrade from 15 -> 16 will ensure any
  conflicting messages are removed from the database.
• improvement: unit tests have been made to verify the above bugfix.