[ZIP 208] [Blossom NU] Shorter Block Target Spacing

[zookozcash]

Who is affected: all users
What: shorter target blocktimes
Why: faster finalisation [*] and greater throughput of transactions per hour.

[*] It's controversial whether this change actually provides this benefit.

Data:

• BTC has 1 MB blocks, 600 second blocktimes, [please reply and insert typical BTC full block verification time here] and has often sustained full load without failing.
• LTC has 1 MB blocks, 150 second blocktimes, but I don't know if it has ever sustained full load.
• ETH has sustained full load at about 20 KB blocks about every 15 seconds, and according to according to Frederick with verification time around 300 ms. (Edit: after other changes to the difficulty in ETH, the average block time for ETH is currently 12.5 seconds.)
• BCH may have sustained load during "stress tests" on their mainnet — I'm not sure about that. Please reply and add that data.
• ZEC (1.0 “Sprout“ and 2.0 “Sapling”) have 2 MB max block size, 150 second block times, estimated worst-case verification time of maybe 20s (??? Please reply and add this data), but to my knowledge we've never sustained full load, not even on testnet or a simulation.

Expert opinion: I've heard from Bitcoin Core developers, Vitalik Buterin, Arthur Breitman, James Prestwich, Daira Hopwood, and Emin Gün Sirer, and probably from lots of other well-informed people who I'm momentarily forgetting (sorry), and they seem to be in rough agreement that if the worst-case block verification time (i.e. the time for a miner to verify a winning block that a different miner just supplied to them, and which that other miner might have maliciously constructed with their own transactions designed to be slow to verify) is no more than 10% of the target block time, then you're probably okay.

Proposal:

1. Write benchmarks of worst-case block verification time.
2. Wire them up to https://speed.z.cash so that we can all see their current results (on a typical 64-bit modern multicore Intel/AMD machine).
3. Optimize the verification code (batch verification of the SNARKs is allowed, SIMD implementation is allowed, multi-core and multi-threading are allowed).
4. Change the target blocktime to be 10X the new worst-case block verification time.

[str4d]

@gtank points out that we also need to model how an increase of users of 2x, 10x, etc. (e.g. due to Sapling light wallets being deployed) can be handled by the current block size and interval. Knowing at what point the chain reaches its limits will tell us how important it is to make these changes.

[defuse]

Here's the ticket where we discussed the block time originally: #764

@ebfull made a good point on that ticket which I'll repeat here to make sure it's not accidentally forgotten:

Reminde[r]: when we do this, we must modify the founders' reward code, because it is based on the assumption we have 10 minute blocks, and thus that the "first four years" end on block 210000 -- the first halving of the block reward.

[nathan-at-least]

I want this renamed to reflect what it means for users. I think that might be "lower the time that users need to wait for finalization and/or increase txn throughput".

[daira]

I had been assuming that the worst case verification time would be for a block full of Sapling spends. However, it might actually be for a block full of JoinSplits if those are reduced to 400 bytes by #3677. (Either Spends or JoinSplits can use dummy inputs, so a DoS attacker wouldn't necessarily need Sapling or Sprout notes to start with.)

[mms710]

Notes from discussion today: we have reached agreement that this is something we'd like to do and on the general approach above, we just haven't yet agreed on what the new target blocktime should be. Good thing the first step of this is to do research and benchmarking!

[zookozcash]

Update: after writing #3690 (comment), I later talked to Dan Robinson, who pointed out that miners already do optimistic verification anyway, which means maybe 10X is far too conservative. Today I asked Zcash Technical Advisory Board member Joseph Bonneau about this goal, and he also independently suggested this same idea — that miners do optimistic verification, so we do not need to reason about miners delaying mining until they've finished verification of the incoming block.

Here's optimistic block verification:

You're a miner. You have three threads: the block-acceptor, the transaction-acceptor, and the PoW-cruncher. The PoW-cruncher is always running fult tilt at all times, because $$$. Occasionally the transaction-acceptor says to the PoW-cruncher "Hey I built a new candidate block for you, buddy, with more better txns. Here you go.", and the PoW-cruncher switches to trying to mine that candidate block instead of the candidate block it was previously trying to mine. Now here's optimistic verification: occasionally the block-acceptor says "Hey I received a new block from someone, and I already verified its PoW and the difficulty, but I haven't yet verified the correctness of all its transactions, go ahead and get started". When he hears this, the PoW-cruncher starts trying to mine an empty candidate block on top of that new block. Once the block-acceptor tells the transaction-acceptor that the contents of the incoming block are all valid, then the transaction-acceptor starts processing transactions and supplying non-empty candidate blocks to the PoW-cruncher. It is possible (but exceedingly rare in practice), that the block-acceptor might then say "Whoa whoa whoa, that incoming block I gave you? The one with the valid PoW and the right difficulty? Turns out it contains invalid transactions. Forget about it and go back to this one instead."

As far as I understand, all miners already do this.

A consequence of this architecture is that having the worst-case block-verification-time be 50%, or 100% or 200% of the target block time might not cause any systemic problem or any vulnerability.

I propose that we initiate careful analysis of the potential systemic problems and the potential vulnerabilities at different worst-case block-verification-time levels (e.g. 50%, 100%, or 200% of the target block time), but in parallel we proceed with defining the target block time in Blossom to be 3X of the estimated worst-case block-verification-time. This seems to me to be a good trade-off of a generous safety margin with a good capacity increase.

[zookozcash]

My current proposal is:

• Write benchmarks of worst-case block verification time.
• Wire them up to https://speed.z.cash so that we can all see their current results (on a typical 64-bit modern multicore Intel/AMD machine).
• Optimize the verification code (batch verification of the SNARKs is allowed, SIMD implementation is allowed, multi-core and multi-threading are allowed).
• Change the target blocktime to be 3X the new worst-case block verification time.

This means that if the worst-case block can't be verified any faster than 50s then we can't increase the block rate at all! So fire up those optimisers, folks! :-)

Ooh, fun idea, possibly a bad idea: have two teams in competition! The "generate a worst-case block" team and the "optimize the verifier" team. We could even have the two teams submit their latest submissions blinded from one another in multiple rounds. :-) ❤

[zookozcash]

Important note about the timing of this: we have to specify everything about the faster blocks in the January 22 specification, but we don't have to choose the constant for the new target block time until the end of March (in time for the code implementation audit). So we have a little less than three months to determine the answer!

If we don't determine this empirically by the end of March, then I propose we reduce the target block time from 150 seconds to 60 seconds. Rationale for that is that it seems roughly in keeping with the empirical evidence from other blockchains (listed in the O.P.), and 60 seconds is a nice easy constant for humans to remember and work with.

If you think 60 seconds is too low then go generate a worst-case block which takes longer than 20 seconds for the current verifier to verify. :-)

[Eirik0]

Closing this ticket because ZIP208 has been merged and we are currently working on implementing it in zcashd v2.0.7.

[bitzec]

Was all this work worth it at all.. ? Sounds like zec needs faster transactions .. whats the reason for this anyway ? I dont get it .. marketing ?

[petertodd]

[Edit by @daira to restore the full quotation.]

Expert opinion: I've heard from Bitcoin Core developers, Vitalik Buterin, Arthur Breitman, James Prestwich, Daira Hopwood, and Emin Gün Sirer, and probably from lots of other well-informed people who I'm momentarily forgetting (sorry), and they seem to be in rough agreement that if the worst-case block verification time (i.e. the time for a miner to verify a winning block that a different miner just supplied to them, and which that other miner might have maliciously constructed with their own transactions designed to be slow to verify) is no more than 10% of the target block time, then you're probably okay.

(emphasis mine)

@zookozcash I'm curious, who did you talk too?