New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testnet becomes unusable after large hashrate drop #3552

Closed
garethtdavies opened this Issue Sep 30, 2018 · 5 comments

Comments

@garethtdavies

garethtdavies commented Sep 30, 2018

Since the removal of the testnet-only difficulty rules, which as per Bitcoin would allow mining of a min-difficulty block after 2 * 2.5 mins (333ea3c) the testnet doesn't have an easy way to recover from a dramatic drop in hashrate and can lead to extended periods of the testnet not finding blocks.

Currently, the testnet is stuck in such a state with the network hashrate at over 60k Sol/s (which has obviously long-since departed) where it previously hovered around ~20 Sol/s.

{
"blocks": 299179,
"currentblocksize": 0,
"currentblocktx": 0,
"difficulty": 262594.272944737,
"errors": "WARNING: check your network connection, 0 blocks received in the last 4 hours (96 expected)",
"genproclimit": 1,
"localsolps": 0,
"networksolps": 60783,
"networkhashps": 60783,
"pooledtx": 0,
"testnet": true,
"chain": "test",
"generate": false
}
@daira

This comment has been minimized.

Show comment
Hide comment
@daira

daira Sep 30, 2018

Contributor

I agree something must be done promptly to fix the problem on testnet, in order for it to be usable for Sapling testing.

As noted by @str4d in the commit comment for 333ea3c, it's actually tricky to apply the Bitcoin testnet's min-difficulty rule to Zcash due to the change in difficulty adjustment algorithm.

We also need to consider compatibility issues; it's acceptable to cause a testnet fork (since it's currently unusable), but we need to be sure that nodes that upgrade after the fork will correctly reorg to the new chain, which is not obvious when the min-difficulty rule is involved if the node has seen blocks after the fork on the old chain.

Contributor

daira commented Sep 30, 2018

I agree something must be done promptly to fix the problem on testnet, in order for it to be usable for Sapling testing.

As noted by @str4d in the commit comment for 333ea3c, it's actually tricky to apply the Bitcoin testnet's min-difficulty rule to Zcash due to the change in difficulty adjustment algorithm.

We also need to consider compatibility issues; it's acceptable to cause a testnet fork (since it's currently unusable), but we need to be sure that nodes that upgrade after the fork will correctly reorg to the new chain, which is not obvious when the min-difficulty rule is involved if the node has seen blocks after the fork on the old chain.

@daira

This comment has been minimized.

Show comment
Hide comment
@daira

daira Sep 30, 2018

Contributor

Arguably, the min-difficulty time threshold should be set so that min-difficulty blocks occur rarely. For twice the block time they occur frequently in "normal operation" due to the exponential distribution of interarrival times. I suggest 6 times the block time, or 15 minutes. (This is independent of the other issues in my previous comment.)

Contributor

daira commented Sep 30, 2018

Arguably, the min-difficulty time threshold should be set so that min-difficulty blocks occur rarely. For twice the block time they occur frequently in "normal operation" due to the exponential distribution of interarrival times. I suggest 6 times the block time, or 15 minutes. (This is independent of the other issues in my previous comment.)

@mms710 mms710 added this to Current Sprint in Zcashd Team Oct 1, 2018

@daira

This comment has been minimized.

Show comment
Hide comment
@daira

daira Oct 3, 2018

Contributor

@str4d and I simulated the effect of large (99%) drops in hash power on a Zcash-like testnet, with no min-difficulty rule and with a min-difficulty threshold time set to 2, 4, 6, or 20 times the target block time. My intuition in the previous comment was correct: 6 times the target block time (i.e. 15 minutes) is long enough to make min-difficulty blocks infrequent in normal operation, but allow recovery from a large hash power drop within a few hours.

This did not seem to cause any significant problem with chain splits or orphans (the simulation is of a network of 100 nodes; and takes into account block propagation and reorgs; we had to modify the calculation of chain work added by a new block to make it more realistic). I'll add graphs of the results later.

I see no obstacle to adding this rule on testnet for 2.0.1. (It is not suitable for mainnet because miners would have an incentive to postdate timestamps.)

Contributor

daira commented Oct 3, 2018

@str4d and I simulated the effect of large (99%) drops in hash power on a Zcash-like testnet, with no min-difficulty rule and with a min-difficulty threshold time set to 2, 4, 6, or 20 times the target block time. My intuition in the previous comment was correct: 6 times the target block time (i.e. 15 minutes) is long enough to make min-difficulty blocks infrequent in normal operation, but allow recovery from a large hash power drop within a few hours.

This did not seem to cause any significant problem with chain splits or orphans (the simulation is of a network of 100 nodes; and takes into account block propagation and reorgs; we had to modify the calculation of chain work added by a new block to make it more realistic). I'll add graphs of the results later.

I see no obstacle to adding this rule on testnet for 2.0.1. (It is not suitable for mainnet because miners would have an incentive to postdate timestamps.)

@daira

This comment has been minimized.

Show comment
Hide comment
@daira

daira Oct 3, 2018

Contributor

Note that an adversarial miner with almost all of the hash power can still cause a delay of an hour by postdating blocks (since honest miners' blocks won't satisfy the increasing timestamp rule until the current time catches up), but that's a separate pre-existing issue.

Contributor

daira commented Oct 3, 2018

Note that an adversarial miner with almost all of the hash power can still cause a delay of an hour by postdating blocks (since honest miners' blocks won't satisfy the increasing timestamp rule until the current time catches up), but that's a separate pre-existing issue.

@str4d str4d moved this from Current Sprint to In Progress in Zcashd Team Oct 3, 2018

@daira

This comment has been minimized.

Show comment
Hide comment
@daira

daira Oct 4, 2018

Contributor

@str4d wrote:

Simulation of the current Zcash algorithm with a 99% multipool miner disappearing (equivalent to what we saw on testnet going from ~60k Sol/s to a couple of GPUs).

Red is difficulty; green is block interval averaged over 10-block window; blue is block interval averaged over 100-block window. Red line uses left-hand Y-axis; green and blue use right-hand Y-axis.

diffalgo-testnet-99pc-nomindiff

Simulation of the Zcash difficulty algorithm with additional min-difficulty blocks if the last block was 15 minutes ago (6x the target block interval):

diffalgo-testnet-99pc-mindiff-6x

Note that the latter graph spikes to 900s, i.e. 15m as expected.

Contributor

daira commented Oct 4, 2018

@str4d wrote:

Simulation of the current Zcash algorithm with a 99% multipool miner disappearing (equivalent to what we saw on testnet going from ~60k Sol/s to a couple of GPUs).

Red is difficulty; green is block interval averaged over 10-block window; blue is block interval averaged over 100-block window. Red line uses left-hand Y-axis; green and blue use right-hand Y-axis.

diffalgo-testnet-99pc-nomindiff

Simulation of the Zcash difficulty algorithm with additional min-difficulty blocks if the last block was 15 minutes ago (6x the target block interval):

diffalgo-testnet-99pc-mindiff-6x

Note that the latter graph spikes to 900s, i.e. 15m as expected.

zkbot added a commit that referenced this issue Oct 5, 2018

Auto merge of #3559 - str4d:3552-testnet-min-difficulty-blocks, r=bit…
…cartel

Allow minimum-difficulty blocks on testnet

This is a consensus rule change on testnet that will result in a chain split (leaving the stuck chain, as desired).

Reverts #2766 and part of #1338.
Closes #3552.

@zkbot zkbot closed this in #3559 Oct 6, 2018

Zcashd Team automation moved this from In Progress to Complete Oct 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment