Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip BIP 30 verification where not necessary #6931

Merged
merged 2 commits into from Nov 12, 2015
Merged

Conversation

@morcos
Copy link
Member

morcos commented Nov 3, 2015

Since BIP 34 became active, it has been impossible to generate duplicate coinbases, and therefore not possible to generate any other duplicate transaction ID's.

This pull will allow us to skip BIP 30 verification if we know we are on a chain after the point BIP 34 was activated and there are no latent duplicate transaction chains.
On mainnet there were 2 cases of duplicate coinbases, and in both cases the duplicate overwrote the original before there was an opportunity to spend it.
I have not actually verified that there are no duplicate coinbases on testnet3, and we need to verify that or turn off the skipping on testnet before merging this pull. (EDIT: verified)

In conjunction with #6932 this will help ConnectBlock be much more efficient with caching access to the database.

@instagibbs
Copy link
Member

instagibbs commented Nov 3, 2015

Do you have a benchmark with both PRs together?

utACK, aside from the testnet3 coinbase issue

@gavinandresen
Copy link
Contributor

gavinandresen commented Nov 3, 2015

Concept ACK, but the logic seems more complicated than necessary. Couldn't you just remove the BIP30 checking code entirely now that BIP34 has activated? Why do the work in the old chain, when we know the old chain doesn't violate the rules (and have headers-first to prevent somebody from feeding us a low-work bogus chain)?

@sdaftuar
Copy link
Member

sdaftuar commented Nov 3, 2015

@gavinandresen Headers-first doesn't actually prevent someone from doing the attack -- for example the single peer a node chooses to download headers from could feed a bogus chain before a node ever learns about the real chain.

Concept ACK

@morcos
Copy link
Member Author

morcos commented Nov 3, 2015

@instagibbs I'm working on putting together some benchmarks. There are a set of pulls that I've been working on to speed up ConnectBlock and CreateNewBlock. I'll post an issue identifying them all and with some benchmarks. But roughly speaking we're talking about these times on average over some recent data for CreateNewBlock (with 1GB dbcache and 1M sigcachesize)

Code version time in ms
Master 550
libsecp256k merged 340
libsecp and these 2 pulls* 160

* I actually didn't do the coinbase lookup required in 6932 for this test, so that can be the difference between 0 and 1 db lookup.

@gavinandresen I'd be interested if you have any thoughts on how best to benchmark this as well?

@gmaxwell
Copy link
Contributor

gmaxwell commented Nov 3, 2015

utACK (will test)

@sipa
Copy link
Member

sipa commented Nov 4, 2015

@TheBlueMatt
Copy link
Contributor

TheBlueMatt commented Nov 6, 2015

Some random stats (generated using #6965):

2015-11-06 23:22:03 UpdateTip: new best=00000000000000000e168ce1bbcdab4551cc35e9d5128b75e5d603143cdc970a height=357552 log2_work=82.822239 tx=69465120 date=2015-05-22 12:42:52 progress=0.799217 cache=1120.9MiB(601684tx)
2015-11-06 23:22:03 - Connect postprocess: 3.40ms [2.65s]
2015-11-06 23:22:03 - Connect block: 328.16ms [230.41s]
2015-11-06 23:22:03 - Load block from disk: 18.28ms [21.80s]
2015-11-06 23:22:03 - Sanity checks: 2.83ms [1.73s]
2015-11-06 23:22:03 - Fork checks: 68.39ms [38.97s]
2015-11-06 23:22:03 - Connect 1501 transactions: 178.28ms (0.119ms/tx, 0.043ms/txin) [92.67s]
2015-11-06 23:22:03 - Verify 4109 txins: 245.83ms (0.060ms/txin) [150.33s]
2015-11-06 23:22:03 - Index writing: 0.04ms [0.06s]
2015-11-06 23:22:03 - Callbacks: 0.03ms [0.03s]
2015-11-06 23:22:03 - Connect total: 318.09ms [191.65s]
2015-11-06 23:22:03 - Flush: 26.23ms [14.01s]
2015-11-06 23:22:03 - Writing chainstate: 2.05ms [0.67s]

2015-11-06 23:32:37 UpdateTip: new best=00000000000000000e168ce1bbcdab4551cc35e9d5128b75e5d603143cdc970a height=357552 log2_work=82.822239 tx=69465120 date=2015-05-22 12:42:52 progress=0.799210 cache=1120.9MiB(601684tx)
2015-11-06 23:32:37 - Connect postprocess: 5.06ms [2.85s]
2015-11-06 23:32:37 - Connect block: 198.12ms [160.29s]
2015-11-06 23:32:37 - Load block from disk: 14.21ms [7.67s]
2015-11-06 23:32:37 - Sanity checks: 2.79ms [1.81s]
2015-11-06 23:32:37 - Fork checks: 0.04ms [0.14s]
2015-11-06 23:32:37 - Connect 1501 transactions: 173.07ms (0.115ms/tx, 0.042ms/txin) [96.87s]
2015-11-06 23:32:37 - Verify 4109 txins: 196.50ms (0.048ms/txin) [131.39s]
2015-11-06 23:32:37 - Index writing: 0.08ms [0.07s]
2015-11-06 23:32:37 - Callbacks: 0.07ms [0.03s]
2015-11-06 23:32:37 - Connect total: 200.48ms [134.03s]
2015-11-06 23:32:37 - Flush: 27.12ms [15.16s]
2015-11-06 23:32:37 - Writing chainstate: 2.27ms [0.83s]

@TheBlueMatt
Copy link
Contributor

TheBlueMatt commented Nov 6, 2015

In other words, of some random set of blocks (~356331-357552), this pull took the ConnectBlock time from 230s to 160s!

@sipa
Copy link
Member

sipa commented Nov 7, 2015

Concept & code review ACK.

@petertodd
Copy link
Contributor

petertodd commented Nov 10, 2015

utACK modulo duplicate testnet coinbases issue.

@petertodd
Copy link
Contributor

petertodd commented Nov 10, 2015

Just confirmed that all coinbases from block #0 to block #21112 on testnet3 are unique.

Also successfully did a sync from scratch on testnet3.

ACK

@sipa
Copy link
Member

sipa commented Nov 11, 2015

Untested ACK

@morcos
Copy link
Member Author

morcos commented Nov 12, 2015

Also confirmed unique coinbases on tesnet3.

@sipa sipa merged commit 33c90cf into bitcoin:master Nov 12, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
sipa added a commit that referenced this pull request Nov 12, 2015
33c90cf Make skipping BIP30 check chain agnostic (Alex Morcos)
06d81ad Skip BIP30 check after BIP34 activation (Alex Morcos)
FireWalkerX added a commit to Crowndev/crown-core that referenced this pull request Oct 11, 2016
Skip BIP 30 verification where not necessary bitcoin#6931
@evoskuil

This comment has been minimized.

Copy link

evoskuil commented on 06d81ad Dec 11, 2016

Once BIP34 activated it was not possible to create new duplicate coinbases and thus other than starting with the 2 existing duplicate coinbase pairs, not possible to create overwriting txs.

This is an incorrect assumption. BIP30 addresses hash collision. It is possible for transaction hashes to collide through nothing more than happenstance.

If we're on the known chain at height greater than 227931 where BIP34 activated, we can save the db accesses needed for the BIP30 check.

BIP34 does not address the real possibility of transaction hash collision, it only addresses duplicate transactions. Eliminating the BIP30 check is a hard fork that will result in the arbitrary destruction of unspent outputs.

Incidentally, the implementation assumes incorrectly that BIP34 must have activated in the case where a checkpoint at which BIP34 previously activated is found to be present. It is possible, however unlikely, that the checkpoint represents a different history, again owing to the possibility of hash collision. Checkpoints are optimizations and should never be used for security.

cerebrus29301 pushed a commit to cerebrus29301/crowncoin that referenced this pull request Feb 4, 2017
bitcoin#6931
BIP-0034 should've been active by the time crown started.
PLEASE TEST.
Former-commit-id: c765f36
cerebrus29301 pushed a commit to cerebrus29301/crowncoin that referenced this pull request Feb 4, 2017
Skip BIP 30 verification where not necessary bitcoin#6931

Former-commit-id: dbea57a
laanwj added a commit that referenced this pull request Mar 7, 2018
5b8b387 Fix overly eager BIP30 bypass (Alex Morcos)

Pull request description:

  In #6931 we introduced a possible consensus breaking change by misunderstanding how completely BIP 34 obviated the need for BIP 30.  Unfixed, this could break consensus after block height about 1.9M.  Explained in code comment.

  h/t @sdaftuar

Tree-SHA512: 8f798c3f203432fd4ae1c1c08bd6967b4a5ec2064ed5f6a7dcf3bff34ea830952838dd4ff70d70b5080cf4644f601e5526b60456c08f43789e4aae05621d9d6b
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Feb 3, 2020
Summary:
In D2076 the check=true parameter was passed, which doesn't just
silently hide overwrites but also causes an extra HaveCoin lookup
for each coin. This is a whole database lookup regardless of the
caches, since it's always going to be a miss in normal operation.
We're better off just catching the exception than doing the silent
overwrite thing.

As a quick benchmark, I ran reindex-chainstate on testnet without/with
this change, and the runtime changed from 29 minutes to 20 minutes.
And that's even with the coins db on an SSD and probably fully in
the OS cache.

I found the comment a bit confusing:
- CheckBlock does not check for duplicate transactions except the
  special case of the merkle tree vuln.
- BIP30 is nice but it's not being enforced on recent blocks (again,
  because HaveCoin misses are slow), only BIP34 is enforced. -- see
  bitcoin/bitcoin#6931

Test Plan:
`ninja check-extended`

(since CTOR is always enabled on regtest, and also regtest always
enforces BIP30, it is unfortunately not possible to write a functional
test for this.)

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D5081
furszy added a commit to PIVX-Project/PIVX that referenced this pull request Aug 2, 2020
c894e8f [Core] Remove BIP30 check (random-zebra)

Pull request description:

  A bit of history
  ----

  Two transactions can have the same txid if their parents are identical, since the txids of the parents are included in a transaction.
  Coinbases have no parents, so it used to be possible for two of them to be identical.
  Further,  by building on duplicate coinbases, duplicate normal transactions were possible as well (http://r6.ca/blog/20120206T005236Z.html).

  In order to remove the possibility of having duplicate transaction ids, Bitcoin introduced, with BIP30, the following consensus rule:
  - Blocks are not allowed to contain a transaction whose identifier matches that of an earlier, not-fully-spent transaction in the same chain. [[1](https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki)]

  This rule was enforced by verifying (in `ConnectBlock`) that none of the transactions in the block was overwriting an already existent non-pruned CCoins entry.

  BIP34 was later added in Bitcoin to enforce better transaction uniqueness, with the update of block version to 2, which introduced the following consensus rule:
  - the first serialized element in the scriptSig of coinbase transactions must be the height of the chain. [[2](https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki)]

  After the complete activation of BIP34, there seemed to be no longer need for the check in `ConnectBlock`, added for BIP30, as duplicated coinbases were rendered impossible with BIP34 (bitcoin#6931).

  This assumption was later revisited, when several blocks were found on Bitcoin's main chain (before BIP34 activation), having coinbase scripts with an height higher than the current chain height (and higher than the BIP34 activation height).
  Thus, coinbases for blocks at these "future" heights would have given the opportunity for BIP30 to be violated even with BIP34 enforced (bitcoin#12204).

  Motivation for this PR
  ----

  PIVX has BIP30 and BIP34 consensus rules already implemented since the chain start.
  The first block after the genesis (height=1) has version 3.
  The "block.nVersion=2 rule" is enforced in `ContextualCheckBlock`
  https://github.com/PIVX-Project/PIVX/blob/af3dd41f86684a5e4a18cfb85d471a199cc980da/src/main.cpp#L3605-L3612

  However the code still has the (somewhat expensive) BIP30 check in `ConnectBlock`, which wasn't needed at all, given the full enforcement of BIP34 since the start of the chain.

  This PR removes it.

  *Side Note*: Even without BIP34, with Proof-of-Stake, coinbase transactions have empty scriptPubKey (thus unspendable outputs), therefore there would have been no need for BIP30 checks in any case (at least after PoS activation height).

ACKs for top commit:
  furszy:
    Great find! ACK c894e8f.
  Fuzzbawls:
    ACK c894e8f

Tree-SHA512: bc0dec1ef68c05db35ee0a132d0817c851667ce47ba5ec7eae31d29f1a44266f987db4462dc16d2320684bf76a6088c3fdbe77e08a9312ab6a1b7a58b6632661
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

10 participants
You can’t perform that action at this time.