Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of explicit wallet conflicts instead of using mempool #7105

Merged
merged 1 commit into from Dec 1, 2015

Conversation

@sipa
Copy link
Member

sipa commented Nov 26, 2015

This is an alternative approach for #7067. It stores the earlier block which conflicts (directly or indirectly) with transactions. The wallet relies on being told about conflicts, so this may need rescanning to discover conflicts with historical transactions (but those aren't reliably detectable anyway, even with other approaches).

New semantics:

  • A negative confirmation count -N means that there is a block (with N confirms) that directly or indirectly (via other wallet transactions) conflicts with a given transaction.
  • Unconfirmed coins received from self are only considered spendable when they are in the mempool.
@jonasschnelli
Copy link
Member

jonasschnelli commented Nov 27, 2015

Concept ACK.
I think this solution performs much better then #7067.
Have plans to add tests for this soon.

@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

Seems to have a deadlock or infinite loop.

@sipa sipa force-pushed the sipa:realconflicts branch Nov 27, 2015
@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

Fixed 3 bugs (infinite loop, marking the accepted rather than conflicting tx, and not rebroadcasting), and added some debug output.

@sipa sipa force-pushed the sipa:realconflicts branch Nov 27, 2015
@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

All rpc tests pass now, including the txn-doublespend.py one which tests negative confirmations.

@gmaxwell
Copy link
Contributor

gmaxwell commented Nov 27, 2015

I tried this on an old wallet that has a number of conflicts in it... they show as 'confirmations": 0,'. (On master, they show as -1). "So much for tests."

Edit: Reading the code, I think this might be intentional. Bleh. I don't think it's acceptable to silently turn -1 confirmed count transactions to 0s in an upgrade, even if we release note it. I think we need to either find a way around that or force a rescan.

@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

@gmaxwell You'll need to rescan for them to be detected.

I don't mind adding some "fall back" logic to guess historical conflicts, but it can't be accurate, as we have no guaranteed way of knowing about them.

@gmaxwell
Copy link
Contributor

gmaxwell commented Nov 27, 2015

@sipa: I'd be okay with it if it was too conservative, e.g. showing an evicted historical transaction as conflicted.

@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

@gmaxwell It can't work even for spends of outputs that were very recently spent, along with all other outputs of those transactions.

@gmaxwell
Copy link
Contributor

gmaxwell commented Nov 27, 2015

As an aside Concept ACK; I like tracking the conflicts explicitly and knowing the conflict depths will be nice.

@sipa
Copy link
Member Author

sipa commented Nov 27, 2015

Maybe we just need a separate class like "limbo" which means unconfirmed, not known to be conflicted, but also not in the mempool?

@sipa sipa force-pushed the sipa:realconflicts branch 2 times, most recently Nov 27, 2015
@sipa
Copy link
Member Author

sipa commented Nov 28, 2015

Added release notes about this change.

@gmaxwell
Copy link
Contributor

gmaxwell commented Nov 28, 2015

Limbo sounds useful. Damn I love this patch, but do not love the inaccurate data on a non-rescanned wallet.

@sipa
Copy link
Member Author

sipa commented Nov 28, 2015

Added a "trusted" field to listtransactions output for unconfirmed transactions.

@sipa sipa force-pushed the sipa:realconflicts branch Nov 28, 2015
@sipa sipa force-pushed the sipa:realconflicts branch to 9ac63d6 Nov 29, 2015
@sipa
Copy link
Member Author

sipa commented Nov 29, 2015

Rebased.

@sipa sipa added this to the 0.12.0 milestone Nov 30, 2015
@sipa
Copy link
Member Author

sipa commented Nov 30, 2015

Tagging as 0.12: this fixes the problem that evicted wallet transaction's inputs are automatically considered respendable.

@laanwj laanwj added the Bug label Nov 30, 2015
@laanwj
Copy link
Member

laanwj commented Nov 30, 2015

Yes, should definitely be in 0.12, should be mentioned that this a fix for a bug where the wallet silently re-spends old transactions because they've been evicted from the mempool, or if they've never been in the mempool (-walletbroadcast).

Edit: The UI regards everything with <0 confirms as 'conflicted', it does not distinguish between different negative values. This is good, it needs no change for this.

@sipa
Copy link
Member Author

sipa commented Nov 30, 2015

@laanwj It may make sense to distinguish in the UI between 'unconfirmed' (meaning in mempool) and something else like 'limbo' (meaning not in mempool)? RPC exposes this via the "trusted" field.

@jonasschnelli
Copy link
Member

jonasschnelli commented Nov 30, 2015

Tested ACK.

Extended the PR with two commits:
https://github.com/jonasschnelli/bitcoin/commits/2015/11/sipa_realconflicts
(mempool limit / eviction RPC test + GUI transactions details improvement).

bildschirmfoto 2015-11-30 um 16 51 07

bildschirmfoto 2015-11-30 um 16 45 54

@dgenr8
Copy link
Contributor

dgenr8 commented Nov 30, 2015

The network is trying to advise wallet immediately of wtx conflicts. We should not stick our fingers in our ears and sing. These should be added to the wallet rather than showing up only when confirmed.

@sipa
Copy link
Member Author

sipa commented Nov 30, 2015

So what do you propose?

  • Pass every mempool rejected transaction to the wallet? That would
    trivially allow dust attacking your system.
  • Only pass rejected transactions that are not rejected because of validity
    or policy? Now it's trivial to bypass.

Yes, more information about double spends is useful, but using the P2P
network for it is only useful for cases where it wasn't intented as an
attack. And those case exist too.

Now, can we please talk about the bugs that this patch is intended to
address? The incorrect assumption that whatever coins aren't spent by the
mempool can safely be considered available for other transactions. That too
will reduce (accidental) double spends.

@dgenr8
Copy link
Contributor

dgenr8 commented Nov 30, 2015

I think this change is good and don't mean to go off topic. It does move a little toward more historical info and less real-time info, by not marking unconfirmed conflicts as visibly. There is a lot going on in those few minutes. It's good to see the wallet providing more information and hopefully soon, more control.

Since you asked, I think the first non-clone conflict of a mempool tx should be relayed, and added to the wallet if the wallet conflicts, with some protections.

@laanwj
Copy link
Member

laanwj commented Dec 1, 2015

utACK

@laanwj laanwj merged commit 9ac63d6 into bitcoin:master Dec 1, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
laanwj added a commit that referenced this pull request Dec 1, 2015
9ac63d6 Keep track of explicit wallet conflicts instead of using mempool (Pieter Wuille)
Warrows added a commit to Warrows/PIVX that referenced this pull request Jul 19, 2019
Warrows added a commit to Warrows/PIVX that referenced this pull request Jul 19, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Jul 20, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Jul 28, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Jul 28, 2019
And add some forgotten changes from
bitcoin#7105 backport
Warrows added a commit to Warrows/PIVX that referenced this pull request Aug 4, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Sep 11, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Sep 23, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
Warrows added a commit to Warrows/PIVX that referenced this pull request Oct 9, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
random-zebra added a commit to PIVX-Project/PIVX that referenced this pull request Oct 9, 2019
91b48c7 [Build] Add new merkle files to CMake lists (warrows)
48a3aff [Wallet] Ignore coinbase and zc tx "conflicts" (warrows)
3572354 [Wallet] Fix an error in tx depth computation (warrows)
6928369 [Tests] Enable abandonconflict functional test (warrows)
34cd496 Fix that CWallet::AbandonTransaction would only traverse one level (Ben Woosley)
aba5b75 Fix calculation of balances and available coins. (Alex Morcos)
48d705f Remove stale wallet transactions on initial load. (presstab)
12985ae Flush wallet after abandontransaction (Alex Morcos)
8f87956 [Wallet] sort pending wallet transactions before reaccepting (dexX7)
9c2f445 [Wallet] Call notification signal when a transaction is abandoned (Jonas Schnelli)
778ebf3 Add new rpc call: abandontransaction (Alex Morcos)
0e86c3e Make wallet descendant searching more efficient (Alex Morcos)
d0083a8 Make sure conflicted wallet tx's update balances (Alex Morcos)
6a50e03 [Wallet] Keep track of explicit wallet conflicts instead of using mempool (warrows)
7ccb2b5 [Wallet] Do not flush the wallet in AddToWalletIfInvolvingMe(..) (warrows)
47345be [Refactor] Move wallet functions out of header (warrows)
ab9efb8 [Wallet] Switch to a constant-space Merkle root/branch algorithm (warrows)
5447622 [Wallet] Do not store Merkle branches in the wallet (warrows)

Pull request description:

  This pull request is a happy melting pot of improvements regarding transactions handling. Most of them are backports from bitcoin. I advise reviewers to check the code of the different commits independently to understand them more easily. However, testing is probably better done all at once.
  I am making a single pull request because these changes are all entangled and introducing some of them without others would probably introduce temporary bugs.

  ## Commits details ##
  - 6c3e2ac backport of bitcoin#6550
  - 5304fdf backport of bitcoin#6508
  - c3eeeac simple code move from the header to the cpp file. It contains no functional change.
  - 6cc4d37 backport of bitcoin#4805
  - 10be1db backport of bitcoin#7105
  - 8a34c32 backport of bitcoin#7306
  - 3caf123, 9e17178 and 240f5b4 are the backport for bitcoin#7312
  - ad6d0b1 backport of bitcoin#5511
  - fcc07c3 backport of bitcoin#9311
  - 5ed5e26 is an update of #825
  - 392d504 backport of bitcoin#7715
  - 7199f3a backport of bitcoin#13652
  - f09d999 enables and fixes the test from bitcoin#7312
  - 4fd43c5 fixes an oversight in bitcoin#7105 backport

ACKs for top commit:
  random-zebra:
    ACK 91b48c7
  Fuzzbawls:
    ACK 91b48c7

Tree-SHA512: 2628cebe98805b8048b920b51ee26fd4f0c53643d78da9b8cb265aede52dfe1d40c8c19d34293c232c5c35be7f1ab89ff5b4a07073a4b27c371ea70eb8708669
CaveSpectre11 added a commit to CaveSpectre11/PIVX that referenced this pull request Dec 14, 2019
mempool

Backport of bitcoin#7105
(9ac63d6)
KolbyML added a commit to KolbyML/Phore that referenced this pull request Feb 2, 2020
KolbyML added a commit to KolbyML/Phore that referenced this pull request Mar 28, 2020
KolbyML added a commit to KolbyML/Phore that referenced this pull request Apr 5, 2020
KolbyML added a commit to KolbyML/Phore that referenced this pull request May 6, 2020
KolbyML added a commit to KolbyML/Phore that referenced this pull request May 6, 2020
wqking added a commit to wqking/Vitae that referenced this pull request May 24, 2020
mempool

Backport of bitcoin/bitcoin#7105
(9ac63d6d3056600c1b784da0e6b98f679fa98b6e)
Kokary added a commit to Kokary/wagerr that referenced this pull request May 26, 2020
Backport of bitcoin/bitcoin#7105
(9ac63d6d3056600c1b784da0e6b98f679fa98b6e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.