Keep track of explicit wallet conflicts instead of using mempool

[sipa]

This is an alternative approach for #7067. It stores the earlier block which conflicts (directly or indirectly) with transactions. The wallet relies on being told about conflicts, so this may need rescanning to discover conflicts with historical transactions (but those aren't reliably detectable anyway, even with other approaches).

New semantics:

• A negative confirmation count -N means that there is a block (with N confirms) that directly or indirectly (via other wallet transactions) conflicts with a given transaction.
• Unconfirmed coins received from self are only considered spendable when they are in the mempool.

[jonasschnelli]

Concept ACK.
I think this solution performs much better then #7067.
Have plans to add tests for this soon.

[sipa]

Seems to have a deadlock or infinite loop.

[sipa]

Fixed 3 bugs (infinite loop, marking the accepted rather than conflicting tx, and not rebroadcasting), and added some debug output.

[sipa]

All rpc tests pass now, including the txn-doublespend.py one which tests negative confirmations.

[gmaxwell]

I tried this on an old wallet that has a number of conflicts in it... they show as 'confirmations": 0,'. (On master, they show as -1). "So much for tests."

Edit: Reading the code, I think this might be intentional. Bleh. I don't think it's acceptable to silently turn -1 confirmed count transactions to 0s in an upgrade, even if we release note it. I think we need to either find a way around that or force a rescan.

[sipa]

@gmaxwell You'll need to rescan for them to be detected.

I don't mind adding some "fall back" logic to guess historical conflicts, but it can't be accurate, as we have no guaranteed way of knowing about them.

[gmaxwell]

@sipa: I'd be okay with it if it was too conservative, e.g. showing an evicted historical transaction as conflicted.

[sipa]

@gmaxwell It can't work even for spends of outputs that were very recently spent, along with all other outputs of those transactions.

[gmaxwell]

As an aside Concept ACK; I like tracking the conflicts explicitly and knowing the conflict depths will be nice.

[sipa]

Maybe we just need a separate class like "limbo" which means unconfirmed, not known to be conflicted, but also not in the mempool?

[sipa]

Added release notes about this change.

[gmaxwell]

Limbo sounds useful. Damn I love this patch, but do not love the inaccurate data on a non-rescanned wallet.

[sipa]

Added a "trusted" field to listtransactions output for unconfirmed transactions.

[sipa]

Rebased.

[sipa]

Tagging as 0.12: this fixes the problem that evicted wallet transaction's inputs are automatically considered respendable.

[laanwj]

Yes, should definitely be in 0.12, should be mentioned that this a fix for a bug where the wallet silently re-spends old transactions because they've been evicted from the mempool, or if they've never been in the mempool (-walletbroadcast).

Edit: The UI regards everything with <0 confirms as 'conflicted', it does not distinguish between different negative values. This is good, it needs no change for this.

[sipa]

@laanwj It may make sense to distinguish in the UI between 'unconfirmed' (meaning in mempool) and something else like 'limbo' (meaning not in mempool)? RPC exposes this via the "trusted" field.

[jonasschnelli]

Tested ACK.

Extended the PR with two commits:
https://github.com/jonasschnelli/bitcoin/commits/2015/11/sipa_realconflicts
(mempool limit / eviction RPC test + GUI transactions details improvement).