Add block blacklisting RPC

[sipa]

This is a rebased version of a patch that mining.bitcoin.cz used during the march 11 2013 hardfork, to be able to continue using 0.8 while still mining the 0.7 chain.

The reason for submitting it to mainline is:

• When implementing this, I found that there were a few edge-cases in the reorganization handling, which are fixed here. They probably won't ever occur in normal operation, but I prefer the code to be robust.
• For emergencies, having a blacklistblock RPC is certainly useful to have in the code, though I prefer not having it in normal releases. It's only enabled when compiling with ENABLE_BLOCK_BLACKLISTING. The RPC code is always compiled, so we can catch refactorings that would break it, though - just the index entry is not present normally.

[BitcoinPullTester]

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/9fe9d788a9eaf40cc5dd1171807b24c9bd170104 for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

[gmaxwell]

Could also be used for whitebox test instrumentation: E.g. have a table with the proper utxo state for every height, and use blacklisting to walk all the way back while checking it.

[jgarzik]

ACK

[mikehearn]

I was thinking it'd be better to allow tx hash blacklisting. To blacklist a block you'd blacklist its coinbase.

TX blacklisting would allow app developers to more easily test double spend and reorg handling in regtest mode.

[sipa]

@mikehearn Makes sense, but that'd be more work, as there is no transaction index (and the optional one isn't used for validation). For blocks, there is already a mechanism for marking them invalid in the block index db.

[mikehearn]

Bitcoin already re-validates the last N hundred blocks on startup, right? Do you imagine your patch ever being used to blacklist a block more than a few hundred blocks deep? If not, then it would still seem to be possible.

[sipa]

Not sure how that is related. This is not something that happens at startup - it can mark a block invalid during execution, and it will reorganize away from it instantly. If anything, this is very useful to test edge cases in the block connection logic.

For blacklisting transactions that are in the blockchain already, you'd need a transaction index anyway, and if you do, it's easy enough to look up the block your transaction is in, and blacklist that. For blacklisting mempool transactions (which is more useful, I think - it's unlikely that you hate a transaction that much that you want to cause a hardfork over it), a different approach is needed, but it'd very simple to do: just call mempool.erase from an RPC call.

[gavinandresen]

Just to clarify: blacklistblock permanently blacklists a block; the only way to undo it is to -reindex the block chain.

I think that should go in the blacklistblock help message; people might assume that the blacklist state is memory-only and can be reset by restarting.

Otherwise: ACK.

[yhaenggi]

ACK (even it isnt up to me), this could save us alot of time if we run in the same trouble again

[wtogami]

It sounds like people are in favor of this but only want the help message to more explicitly explain what it does and the danger of misusing it?

[sipa]

@mikehearn I wrote this pull request as preparation for headers-first sync, and the current headers-first sync pull request still includes it, but most of the code touched here (except the actual RPC implementations) is rewritten for headers-first anyway.

[mikehearn]

Maybe it should be closed then.

[wtogami]

Close?

[laanwj]

So should this be closed?

[sipa]

Closing until updated.