Skip to content
This repository has been archived by the owner on Oct 9, 2019. It is now read-only.

Update to be compatible with latest BIP91 spec #21

Merged
merged 8 commits into from Jun 16, 2017

Conversation

jameshilliard
Copy link

I've modified BIP91 to use a smaller confirmation window and enforce mandatory signalling upon lock-in. This should reduce the chance of a conflict with BIP148.

@opetruzel
Copy link

Excellent modification! ACK.

@ghost
Copy link

ghost commented Jun 14, 2017

Concept ACK

@jgarzik
Copy link

jgarzik commented Jun 15, 2017

Concept ACK - will start throwing some activation tests at this

@udiWertheimer
Copy link

Concept ACK

@tomasvdw
Copy link

tomasvdw commented Jun 15, 2017

If I read this correctly, this means that if a miner disagrees and does not wish to signal (bit 4 or 1) and be counted as support, it will have no upgrade window at all when lock-in happens anyway.

His blocks will be immediately orphaned.

In my opinion, that is not acceptable.

@jameshilliard
Copy link
Author

@tomasvdw That shouldn't be a problem in practice as it should be obvious by which pools are signalling if the threshold will be reached well ahead of lock-in and enforcement. There will also likely be some time between the threshold being reached and lock-in for miners to upgrade. Miners can always run the code but not trigger signalling in their stratum servers until it becomes mandatory.

@hsjoberg
Copy link

Concept ACK

@tomasvdw
Copy link

tomasvdw commented Jun 15, 2017

@jameshilliard But before LOCK-IN happens, a miner signaling is counted towards the 80%.

There will also likely be some time between the threshold being reached and lock-in for miners to upgrade.

I don't think this is the case. LOCK_IN happens at the exact moment the threshold is reached, when 80% of a period is signaled.

Sure they can develop a solution where the signaling is changed automatically at exact LOCK_IN but they shouldn't be required to do so in order not to lose money. This is what LOCK_IN is for.

Is there anything structurally wrong with the timing and activation mechanism of BIP9 of which the LOCK_IN is an important ingredient? Why are we changing it? Is this only to attempt to rush this in before August 1?

@felipelalli
Copy link

felipelalli commented Jun 15, 2017

ACK.

@jameshilliard
Copy link
Author

LOCK_IN happens at the exact moment the threshold is reached, when 80% of a period is signaled.

No, that only changes at the confirmation window boundary.

Is there anything structurally wrong with the timing and activation mechanism of BIP9 of which the LOCK_IN is an important ingredient? Why are we changing it? Is this only to attempt to rush this in before August 1?

The risk to the network from a few miners who weren't paying attention getting their blocks orphaned(most miners not signalling segwit are paying attention enough to manually set the block version already) is not nearly as bad as a potential persistent chain split.

@kek-coin
Copy link

LOCK_IN happens at the exact moment the threshold is reached, when 80% of a period is signaled.

LOCK_IN happens on a difficulty readjust; already before that should it become clear that ~80% of blocks in the period so far are signalling.

Sure they can develop a solution where the signaling is changed automatically at exact LOCK_IN but they shouldn't be required to do so in order not to lose money.

Most softforks have broken status quo mining practices, essentially forcing miners to upgrade or produce invalid blocks, is there a reason that this is now a problem when it didn't use to be?

Is there anything structurally wrong with the timing and activation mechanism of BIP9 of which the LOCK_IN is an important ingredient? Why are we changing it? Is this only to attempt to rush this in before August 1?

Unnecessary chainsplits are not good for anyone involved with Bitcoin, and in this case the change that unsupporting miners have to introduce when signalling reaches 80% is as simple as flipping a single bit in their blocks; they don't even have to implement any mandatory signalling if they don't want to by relying on the 80% hashrate to signal honestly, which is why a 2-week grace period is not as crucial as it is in other softforks.

@jameshilliard
Copy link
Author

LOCK_IN happens on a difficulty readjust; already before that should it become clear that ~80% of blocks in the period so far are signalling.

Here it does not because we are not using 2016 block periods, it instead happens at the edge of the 672 block confirmation boundary.

@tomasvdw
Copy link

@jameshilliard , @kek-coin

No, that only changes at the confirmation window boundary.

LOCK_IN happens on a difficulty readjust; already before that should it become clear that ~80% of blocks in the period so far are signalling.

The threshold is defined at confirmation window boundary. Hence the threshold is reached at the moment of LOCK_IN. You can maybe make some estimate based hash-rate beforehand but that is not really relevant.

in this case the change that unsupporting miners have to introduce when signalling reaches 80% is as simple as flipping a single bit in their blocks;

That is a reasonable point, and indeed a difference with other softforks. I do not agree however that this merits a empty lock_in window, as upgrading is still required. Even if this takes a day, you are still threatening miners with losing money if they don't signal. That is a bad precedent, and not in accordance with a normal threshold based activation in the spirit of the agreement.

@jameshilliard
Copy link
Author

The threshold is defined at confirmation window boundary. Hence the threshold is reached at the moment of LOCK_IN.

This would only be true if there were exactly 538 blocks signalling within the confirmation window.

@sneurlax
Copy link

Concept ACK

The best thing for the ecosystem as a whole is compatibility with UASF; we can avoid a train wreck by building our rails wisely.

@homopit
Copy link

homopit commented Jun 15, 2017

Concept NACK

To start orphaning blocks without any upgrade window after lock-in is not acceptable. Even more, orphaning blocks is really not the goal of NY agreement.

@gavinandresen
Copy link

Code changes look good to me.

@kek-coin
Copy link

kek-coin commented Jun 15, 2017

Even if this takes a day, you are still threatening miners with losing money if they don't signal. That is a bad precedent, and not in accordance with a normal threshold based activation in the spirit of the agreement.

The alternative (bigger risk of a chainsplit) would be in my opinion a far more dangerous precedent to set. I have to admit I am not privy to the spirit of the agreement, but it seems to me that showing good faith by promoting a unified blockchain is within its boundaries; I don't particularly see how anything in the agreement specifies that some 20% of miners won't be orphaned if they refuse to jump on board with a hardfork either.

Edit: to clarify, I mean that there is still a threat for miners to lose money, as there has been with most softforks in the past as well. This "losing money" threat seems to be par for the course of Bitcoin, I don't see why measures that introduce a way to keep the chain unified should be disregarded because of this - unless there is a better way to avoid a chainsplit that does not entail such risks.

@jgarzik
Copy link

jgarzik commented Jun 15, 2017

Factually speaking, (a) there is a behavior disparity in the state between 80% and 95%, and (b) the working presumption of this project is that there will be a hard fork, with all that entails.

@Saicere
Copy link

Saicere commented Jun 15, 2017

Concept ACK.

You would get a tiny grace period anyway unless it is a dead heat with 537 blocks signalling when the last block in the 672-block period is pending - in which case, simply signaling for BIP141 temporarily is quite doable. It's not ideal, but I expect miners to all be awake and watching this very closely, so it is likely worth it to get the increased chance for BIP148 compatibility.

@nitsujlangston
Copy link

Concept ACK.
The overrides are a much cleaner way to handle deployment specific window and threshold.

@earonesty
Copy link

utACK

@shawnholt
Copy link

This is brilliant, elegant and the only reasonable path forward Concept ACK.

@hoffmabc
Copy link

Looks like 3 tests failed in there @jl2012

p2p-versionbits-warning.py
sendheaders.py
p2p-compactblocks.py

@benkloester
Copy link

benkloester commented Jun 16, 2017

@tomasvdw I tend to agree that at least some lock-in time would be nice, but I'm not au fait with exactly how tight the timelines are. Is signalling for BIP91 as part of Segwit2x still only meant to start on July 21?

As it stands, with a 672 block confirmation window, and no gap between lock-in and activation (of BIP91) here is the amount of grace time (defined as time between certainty that lock in will occur, and when orphaning begins) miners would get to switch over to signalling bit 1 to avoid being orphaned.

  • Exactly 80% support = No expected grace period as the 538th block signalling bit 4 is expected to arrive in the very last possible block where it is still possible to lock in, ie 672nd in the relevant window, .

  • 85% support = Expected grace period of ~29 blocks, as the 538th block signalling bit 4 is expected to arrive around the 633rd block in the window.

  • 90% support = Expected grace period of ~75 blocks, as the 538th block signalling bit 4 is expected to arrive around the 597th block in the window.

  • 95% support = Expected grace period of ~106 blocks, as the 538th block signalling bit 4 is expected to arrive around the 566th block in the window.

So the more support it gets above 80% the more time other miners will have to switch to signalling bit 1 on lock-in.

It seems even a lock-in period of eg 75 blocks, around half a day, would have a big effect - the difference being equivalent to bit 4 signalling being at 90% rather than 80%.

OT: Was anyone else mildly amused that the number of blocks needed to lock in matches the number of US electoral college members?

@jameshilliard
Copy link
Author

jameshilliard commented Jun 16, 2017

I'm considering changing BIP91 to use a 336 block confirmation period without mandatory signalling during the lock-in period, I think that should address most concerns here as we still get fast(won't be any slower than it is currently) activation but have a ~2 day grace period then before enforcement.

@jameshilliard
Copy link
Author

I've updated BIP91 to use the smaller confirmation window and changed it so enforcement isn't until activation.

@benkloester
Copy link

benkloester commented Jun 16, 2017

@jameshilliard seems to be pretty close to a Pareto improvement on the previous implementation. Only arguable downside I can think of is confirmation will be subject to more variance.

So more chance it activates due to luck - but also more chance it doesn't activate due to luck.

Edit: Good point, this is true in any given period, but with lots of periods this tends to favour activation as it is irreversible so you only have to be lucky once.

@jameshilliard
Copy link
Author

So more chance it activates due to luck - but also more chance it doesn't activate due to luck.

It's pretty much strictly more chance not less since there are more activation periods, and 80% is a high enough threshold that there will be a solid supermajority regardless of variance.

@tomasvdw
Copy link

@jameshilliard The small confirmation window makes it possible for this to activate with less then 80%, which is not part of the agreement.

I do not understand the motivation. What is there to gain from a few weeks? Changing BIP9 activation just to rush things seems like a bad idea.

Is it really to "get on board" the ~0.3% that wants to fork off at August 1? Isn't that their right to do so? I would think that it is safest to only activate SegWit2X for those that want SegWit2X, because trying to get SegWit also for all SegWit-only proponents only postpones the problems to the more risky HF. This is also why I doubt BIP91 in general, but especially this rushing.

@jameshilliard
Copy link
Author

The small confirmation window makes it possible for this to activate with less then 80%, which is not part of the agreement.

The agreement says "at an 80% threshold" and this fits that, there is always variance when it comes to activation and the activation period here is long enough that we will be reasonably close still to 80% in regards to actual hashpower.

Is it really to "get on board" the ~0.3% that wants to fork off at August 1? Isn't that their right to do so?

It's difficulty to know in advance how many intend to run BIP148, especially since it's largely not based on hashpower percentages.

I would think that it is safest to only activate SegWit2X for those that want SegWit2X, because trying to get SegWit also for all SegWit-only proponents only postpones the problems to the more risky HF.

This has already been discussed in depth, even if one wanted to do this there are technical reasons that make it far more difficult to test properly unless we were to wait until the existing segwit deployment expires.

This is also why I doubt BIP91 in general, but especially this rushing.

There are more than enough blocks in the activation period still to ensure a strong supermajority, I think the risk of any significant issues in regards to a fast BIP91 activation are much less than the risks associated with a potential chain spit.

@benkloester
Copy link

It still requires 80% of 336 blocks, which requires that on average 80% of hashpower be signalling bit 4.

Even at 2016 blocks, your argument that it could activate with less than 80% hashpower is true, so it is a question of degree, not a qualitative difference. Why is one against the agreement and not the other?

Given the likelihood of not just a UASF-instigated chain split, but another hardfork chain split in reaction to that come August 1st, there seems to be a lot to gain from a few weeks.

The biggest concern for me is whether miners will accept this BIP91-compatible Segwit2x, given the possibility of bait-and-switch on the HF after Segwit activates. But that's beyond the scope of this pull request.

@kek-coin
Copy link

@benkloester this bait-and-switch is an inherent risk of combining a softfork and a hardfork, even if the softfork is a new deployment; it is trivial to create a client that incorporates the softfork part (and therefore benefits from Segwit) but not the hardfork part (and therefore benefits from backwards compatibility with the majority of the network).

This has been discussed extensively before, it is just not possible to coerce the network into running hardfork code, it must happen with the carrot not the stick. I think that if the btc1 project (including the involved miners) show the goodwill of not causing a chainsplit on August 1, the hardfork will be less contentious. Of course proper review of the hardfork needs to happen, out in the open, but this PR and the responses to it inspire hope moving forward.

@benkloester
Copy link

I know that, you know that, and miners know that. Which is why I'm worried that they won't wan't to run this. Then again, the NY agreement was explicitly to do "segwit now, hardfork in less than 6 months", so this is technically in line with what 80% of hashrate already agreed to.

Your take has given me some hope that perhaps both sides will act honourably enough that both Segwit activation, and a subsequent hard fork to bigger blocks, will gain majority support.

@jgarzik jgarzik merged commit 572278e into btc1:segwit2x Jun 16, 2017
@jacob-eliosoff
Copy link

Concept ACK. And thanks to @jameshilliard (also @Saicere & others) for constructive discussion, I agree with everything he said.

@painlord2k
Copy link

What will happen when, during the window between SW activation and 2 MB activation, the SegWit supporting miners decide to not activate the Hard Fork?

Because I continue to read stuff like this:

https://twitter.com/giacomozucco/status/876832834819944448
Giacomo Zucco @giacomozucco
1 Miners signaling "NYA" are de-facto only activating bip141, and the "future 2MB HF" narrative is just political marketing to hide defeat.

image

What will be the reaction of the Hard Fork supporting miners?

Do they take it and suck it up?
Do they just stop enforcing SegWit and fork to bigger blocks and to hell with the consequences and whoever was so naive to have his coins segregated from his secret keys.

@Pheromon
Copy link

I think the risk is real.

In this respect, to be in compliance with BIP148 is a bug and not a feature exactly because in this way we can't know the exact quantity of hashing power supporting the HF together with the segwit.

A better course would be to not enable segwit2x before at least 10-15 days after UASF activation, to have enough time to let UASF fail.

Once UASF is out of the game we can proceed with more certainty that the network has been upgraded to segwit2x (and not only segwit) and 2MB HF will be successfull.

@bellaj
Copy link

bellaj commented Jun 20, 2017

ACK

@RHavar
Copy link

RHavar commented Jun 20, 2017

@Pheromon I understand your point, but I think you're looking at it the wrong way. Intentionally making it bip125 incompatible very well might prove that the UASF fails but it'll create a needless hostility going forward, which is going to make the HF harder, not easier.

I (as an operator of one of bitcoins biggest gambling sites) am happy to support SegWit2x in all it's part, but it's important to demonstrate good-will by making it compatible with bip141/bip125. I think my views are representative of the quiet majority, who just an amicable seize-fire of the "scaling debate" while we wait for 2nd-layer solution to mature, instead of it being turned into a cock swinging contest, which I want no part of.

@shawnholt
Copy link

I agree with @RHavar . If you look at the price of BTC it has followed the debate very accurately. When the NYC Consensus was announced the price rose - when the disconnect between segwit2x and USAF other speculation / messages like @painlord2k quotes of FUD would flourish, the price crashed and Alt coins rose. There is finally some trust and faith that segwit2x might actually work and the price is rising. The market needs a solution and as @Pheromon points out the path of least resistance is for minors to signal on bit 1 and bit 4 to allow segwit to lock before August 1st. There could be more gamesmanship and all sorts of other disobedience with the 2MB HF and possibly a chain split but there could also be some compromise and consensus. In fact, if miners use Core to signal bit 1+4, they may participate and allow a 2MB solution. It will certainly put those that followed through on the NYC Segwit2MB agreement on higher ground - forcing a split would be seen as mature follow-through and not hostile. The community simply will not accept a HF without a chain split right now so let's move from perfection to pragmatism and focus on innovating rather than name calling. The perfect is the enemy of the good :)

@Pheromon
Copy link

I understand your point, but I think you're looking at it the wrong way. Intentionally making it bip125 incompatible very well might prove that the UASF fails but it'll create a needless hostility going forward, which is going to make the HF harder, not easier.

We must be realistic: the reality is that UASF supporters do not want to HF the network to augment the block size, and they see the 1 Aug activation as a victory. They already state that they oppose to a conseguent HF. There is no good will from their part (from their own admission), so there is no need to expect any.

Bending s2x to activate before or together with UASF it's simply the shift of a problem: you do not create a (small) problem activating segwit, but you will have possibly a (big) problem activating the consequent HF.

Instead, if you create a little (because of no clear miner support) problem during segwit activation you can activate the following 2MB HF in a much safer way.

My fear is that, like the previous attempt, the agreement will fail and the final result wil only be segwit activation while keeping the ancient 1MB block size forever.

UASF is an abomination, it's contrary to all Bitcoin principles and to PoW, and it should be relegated to some obscure altcoin, please let it fail spectacularly instead of giving it strength being "compatible" with it.

@jacob-eliosoff
Copy link

I think it's pretty simple. Miners are getting behind Segwit2x extremely quickly. Segwit2x will activate, and will activate segwit, probably fast enough to make Aug 1 a non-event. Most miners want the HF and will stick with it. But various others opposed to Segwit2x will disable the HF once segwit is locked in, and try to persuade businesses/miners to do likewise. Most miners are strongly motivated to be on the same chain as other miners, so I expect the vast majority of hashpower will follow through with the HF (assuming no show-stopper tech problems come to light). The big question is how much of the rest of the ecosystem will decline the HF, and minimizing that schism should be our top priority.

@RHavar
Copy link

RHavar commented Jun 20, 2017

UASF is an abomination, it's contrary to all Bitcoin principles and to PoW, and it should be relegated to some obscure altcoin, please let it fail spectacularly instead of giving it strength being "compatible" with it.

While UASF are no doubt reckless, they are very much inline with bitcoin ethos and have been done successfully in the past to force consensus. Anyway, this is very much a tangent to the issue on hand and based on the fact your account is only minutes older than your comment -- I'm guessing your only intention is to muddy the waters.

@Pheromon
Copy link

Pheromon commented Jun 20, 2017

I'm guessing your only intention is to muddy the waters.

You guess wrong: I've to use a throw away account because non-core supporters get attacked and ridiculed by a handful of professional trolls and I can't let this happen due to my profession.

My goal is simply to try to maximize the success of S2X while avoiding to linger stuck with a 1MB block once more, possibly forever.

@gandrewstone
Copy link

gandrewstone commented Jun 21, 2017

I'm skimming the BIP91 but it looks like the signaling is somehow backwards compatible with existing SegWit signaling. So I have to agree with @painlord2k. Although I wasn't there it seems like the intention of the NY agreement was a compromise of a 2MB hardfork for segwit. If both of these do not happen at the same time there's a possibility that the one that activates first will "renege" on fulfilling the rest of the agreement. But at least to do so the miner would have to begin by running NYA code and then switch back, which is a clear violation of the agreement. To make the signaling backwards compatible allows people to never switch to NYA compatible code, and seems to me to practically guarantee that SegWit will activate first and the HF will or won't activate based on whether it has 51% of the hash power at that time. This is no different than what we had before the NYA, and is no compromise whatsoever.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet