Make check to distinguish between orphan txs and old txs more efficient. #10557

Merged
merged 1 commit into from Jul 14, 2017

Conversation

Projects
None yet
6 participants
Contributor

morcos commented Jun 8, 2017

Checking for the existence in the CCoinsViewCache of the outputs of a new tx
will result in a disk hit for every output since they will not be found. On the
other hand if those outputs exist already, then the inputs must also have been
missing, so we can move this check inside the input existence check so in the
common case of a new tx it doesn't need to run.

The purpose of the check is to avoid spamming the orphanMap with slightly old
txs which we have already seen in a block, but it is already only optimistic
(depending on the outputs not being spent), so make it even more efficient by
only checking the cache and not the entire pcoinsTip.

@sipa this is my biggest feedback to #10195

For a single valid tx with 500 outputs in regtest on a machine with an SSD, the time to ATMP dropped from 4 ms to 2.4 ms.

+ // Are inputs missing because we already have the tx?
+ for (size_t out = 0; out < tx.vout.size(); out++) {
+ // Optimistically just do efficient check of cache for outputs
+ if (pcoinsTip->HaveCoinInCache(COutPoint(hash, out))) {
@sipa

sipa Jun 8, 2017 edited

Owner

I believe this risks resulting in true for transactions that are in fact not yet confirmed and not in the mempool, but were just disconnected in a reorg. Is that something we care about? If not, perhaps add a comment to explain.

@sdaftuar

sdaftuar Jun 8, 2017 edited

Contributor

Can we just change HaveCoinInCache() to have the same semantics as HaveCoin(), and only return true if the coin is unspent? Seems confusing that their semantics aren't the same.

For many of the existing usages of HaveCoinInCache(), we store the result to potentially Uncache() later -- but I don't think it's possible to ever have a spent coin in your cache that is able to be uncached (ie not marked DIRTY)?

This issue affects the existing usage of HaveCoinInCache() in AlreadyHave(), I think.

@morcos

morcos Jun 8, 2017

Contributor

@sipa I actually think that's likely a better outcome, but it may be too confusing. If a tx was just disconnected in a reorg, and no longer has one of its inputs available, it means it or its parent was probably double spent and so it shouldn't be in the orphan map anyway.

@sipa

sipa Jun 8, 2017

Owner

@morcos Do you think it's worth the complexity? I like @sdaftuar's suggestion.

@morcos

morcos Jun 8, 2017

Contributor

Yes I like his suggestion as well.
I think this PR is "correct" as is, and I will make that change in a separate PR as a fix to HaveCoinsInCache?

Owner

sipa commented Jun 8, 2017

utACK 301566d

fanquake added the Validation label Jun 8, 2017

Owner

sipa commented Jun 23, 2017

Needs rebase.

@morcos morcos Make check to distinguish between orphan txs and old txs more efficient.
Checking for the existence in the CCoinsViewCache of the outputs of a new tx
will result in a disk hit for every output since they will not be found.  On the
other hand if those outputs exist already, then the inputs must also have been
missing, so we can move this check inside the input existence check so in the
common case of a new tx it doesn't need to run.

The purpose of the check is to avoid spamming the orphanMap with slightly old
txs which we have already seen in a block, but it is already only optimistic
(depending on the outputs not being spent), so make it even more efficient by
only checking the cache and not the entire pcoinsTip.
18bacec
Contributor

morcos commented Jun 27, 2017

Simple rebase due to #10503

Contributor

TheBlueMatt commented Jun 27, 2017

utACK 18bacec

Contributor

morcos commented Jul 7, 2017

Please tag for 0.15

fanquake added this to the 0.15.0 milestone Jul 7, 2017

@gmaxwell

utACK. "Duh"-- it should have always done this.

Member

gmaxwell commented Jul 14, 2017

This will spam the orphan map somewhat after flushing, unfortunately-- but we can fix this by making flushing less hard in the future... and the orphan map is much less important than it used to be already.

Contributor

TheBlueMatt commented Jul 14, 2017

I still think we should take this for 15, but another thing to consider is to add a rolling bloom filter in net_processing ala TheBlueMatt/bitcoin@316516c (which is based on a big pile of changes, but could be pulled out separately), on top of this change.

@sipa sipa merged commit 18bacec into bitcoin:master Jul 14, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@sipa sipa added a commit that referenced this pull request Jul 14, 2017

@sipa sipa Merge #10557: Make check to distinguish between orphan txs and old tx…
…s more efficient.


18bacec Make check to distinguish between orphan txs and old txs more efficient. (Alex Morcos)

Tree-SHA512: b6b4bad89aa561975dce7b68b2fdad5623af5ebcb9c38fd6a72b5f6d0544ed441df4865591ac018f7ae0df9b5c60820cb4d9e55664f5667c9268458df70fd554
66270a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment