Replace the ticket database with an efficient, atomic implementation

[cjepson]

The legacy ticket database, which was GOB serialized and stored on
shut down, has been removed. Ticket state information is now held in
a stake node, which acts as a modularized "black box" to contain all
information about the state of the stake system. Stake nodes are now
a component of the blockchain blockNode struct, and are updated with
them.

Stake nodes, like their internal treap primitives, are immutable
objects that are created with their connect and disconnect node
functions. The blockchain database now stores all information about
the stake state of the best node in the block database. The blockchain
makes the assumption that the stake state of the best node is known at
any given time. If the states of former blocks or sidechains must be
evaluated, this can be achieved by iterating backwards along the
blockchain from the best node, and then connecting stake nodes
iteratively if necessary.

Performance improvements with this new module are dramatic. The long
delays on start up and shut down are removed. Blockchain
synchronization time is improved approximately 5-10x on the mainnet
chain. The state of the database is atomic, so unexpected shut downs
should no longer have the ability to disrupt the chain state.

An upgrade path has been added for version 1 blockchain databases.
Users with this blockchain database will automatically update when
they start their clients.

Thanks to @davecgh for the efficient treap primitive, which is widely
used.

TODO before merge:

• Extend unit test coverage.

Fixes issue #337.

[raedah]

Will this prevent running older versions of the decred software once the database has been upgraded?

[davecgh]

@cjepson can confirm, but I would expect the answer no. The stake database would be recreated from scratch using the old format (assuming it's not still around up to a certain point in which case it would continue from the last known good block).

[chappjc]

Upgrade of a mainnet database to version 2 completed successfully in about 17 minutes.
Graceful shutdown of dcrd is less than a second (can it be?).
Sync of entire mainnet chain in 53 minutes.

BTW, discovered this trick: git fetch origin pull/349/head:fast-ticketdb

[jolan]

I've mined a few dozen testnet blocks with this. No issues encountered (will update if so).

tACK

[cjepson]

@raedah It's not recommended to run the old and new databases from the same directory. I would rename the old data directory and then reference it directly with --datadir if you still wanted to use an old version concurrently.

[jcvernaleo]

I've been running this one both of my testnet nodes for several days (one IBD and one upgraded) without issue.

We did see some rejected blocks on an upgraded bitrig node early on but have had no luck reproducing that failure (and I think that was a few commits ago).

[jolan]

Ran OK for me over the weekend on testnet. Was buying tickets/voting constantly. No problems encountered.

tACK

[jcvernaleo]

My two nodes were also good running over the weekend.

[jolan]

Ran the latest version over the weekend on various testnet nodes that are used for stake pool testing and ticket purchasing and they've been running smoothly. Also have a mainnet node that's mostly idle and that's been running ok too.

[jcvernaleo]

My testnet nodes have also been running fine.

[jrick]

Been reviewing this over the past couple days and have not noticed any critical errors. There may be some performance improvements that can still be made but these can be shaken out in tree.

[jolan]

Re-tested PoW mining on testnet with the latest version. 16 blocks so far without problems.

tACK