Add intervening v5 fork for increased min block size #1869
Conversation
| // version 5 starts from block 1406997, which is on or around the 20th of September, 2017. Fork time finalised on 2016-09-18. | ||
| { 5, 1406997, 0, 1505865600 }, | ||
| // version 5 starts from block 1288616, which is on or around the 15th of april, 2017. Fork time finalised on 2016-03-14. | ||
| { 5, 1288616, 0, 1489520158 }, |
There was a problem hiding this comment.
1489520158 - Tue, 14 Mar 2017 19:35:58 GMT
Maybe:
1492214400 - Sat, 15 Apr 2017 00:00:00 GMT
Reported by iDunk on IRC.
There was a problem hiding this comment.
If amending anyway, please capitalise April
There was a problem hiding this comment.
Wasn't the fork time finalized on 2017 instead of 2016?
There was a problem hiding this comment.
You can disregard my IRC post. Six months from the above date, monerod should start notifying about an upcoming fork. That is what the date is about, So, if v6 is still scheduled for 20th of September, that date is about right.
| @@ -501,9 +501,9 @@ bool simple_wallet::set_default_priority(const std::vector<std::string> &args/* | |||
| else | |||
| { | |||
| priority = boost::lexical_cast<int>(args[1]); | |||
| if (priority != 1 && priority != 2 && priority != 3) | |||
| if (priority < 1 || priority > 4) | |||
There was a problem hiding this comment.
Shouldn't this be priority < 0 || priority > 4 ? According to the message priority 0 seems to be valid.
There was a problem hiding this comment.
The priority 0 case is handled just above here.
| @@ -3320,20 +3320,35 @@ bool wallet2::load_tx(const std::string &signed_filename, std::vector<tools::wal | |||
| return true; | |||
| } | |||
| //---------------------------------------------------------------------------------------------------- | |||
| uint64_t wallet2::get_fee_multiplier(uint32_t priority, bool use_new_fee) const | |||
| uint64_t wallet2::get_fee_multiplier(uint32_t priority, int fee_algorithm) const | |||
There was a problem hiding this comment.
Maybe this can be uint8_t.
src/cryptonote_core/blockchain.cpp
Outdated
| if (median_block_size < CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V2) | ||
| median_block_size = CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V2; | ||
| const uint64_t min_block_size = get_min_block_size(version); | ||
| const uint fee_per_kb_base = version >= 5 ? DYNAMIC_FEE_PER_KB_BASE_FEE_V5 : DYNAMIC_FEE_PER_KB_BASE_FEE; |
There was a problem hiding this comment.
Maybe change uint into unsigned int so that it does not break compiling on Windows?
There was a problem hiding this comment.
Whee. Funny when some autocomplete fail still builds. I've no idea what defines uint. Thanks.
Minimum mixin 4 and enforced ringct is moved from v5 to v6. v5 is now used for an increased minimum block size (from 60000 to 300000) to cater for larger typical/minimum transaction size. The fee algorithm is also changed to decrease the base per kB fee, and add a cheap tier for those transactions which we do not care if they get delayed (or even included in a block).
3396a9f Add intervening v5 fork for increased min block size (moneromooo-monero)
|
Did I miss where this went from @JollyMort's draft proposal to something that was vetted publicly? I saw some discussion at the end of the last dev meeting but it was kind of deemed as no longer on meeting topic from what I could understand, and the meeting was closed while that discussion continued. 300kb blocks is the equivalent of 1.5 MB blocks in Bitcoin... Users with limited bandwidth and data caps are going to be punished if the min block sized is raised this high. Plus, isn't the whole point of the block reward fee applied to bigger mined blocks to create a fee market and prevent spam? If demand exists to raise the block size, tx fees pay for that. If there's an issue with the algorithm governing how this expansion happens, how does changing the minimum block size help at all? Very confused... |
|
This is simpler as it avoids complexity of some new formula and achieves nearly the same thing. The whole problem is because of (typical TX size) / (min. blocksize), so since we can't change TX size, we lift the block size to fix the ratio. This is a "stage 1" fix, and actually I kind of hope it will be enough by itself, we'll see. First time I talked about it with ArticMine we were talking about 500-800kB, and then started thinking about some kind of transition instead.
Even before this fix, there was nothing really stopping blocks from going anywhere. The network could get to 10MB if needed, with or without this fix. But miners still have a veto. They can always choose to mine a smaller block. What dynamic block size is for is to allow them to "quickly" expand the blocks to clear some backlog. It's just that there was this massive resistance to start the process of increase which was never meant to be there and was introduced solely by typical TX size increase as a consequence of RCT. It can be clearly seen in the chart below.
For comparison, here it is together with the same kind of chart for pre-RCT TX-es (right) and in ideal case (left). Even the pre-RCT TX-es had a little spike but it was never significant enough to present a barrier.
The min. do-not-relay fee will still be there as anti-spam. What we're really doing with this fix is jumping over that big, unnatural spike on the first chart and settling somewhere in the "almost normal" zone. To be completely out of the anomaly, we'd really need to get to around 1MB blocks. |
Not stopping it entirely, but there is the cost of increasing it to at least discourage. If there is sufficient demand to justify paying that cost, then sure people with data caps are going to be out of luck. However, that doesn't invalidate the concern over reducing the minimum while removing the cost altogether. To clarify, I'm supportive of this change, or at least neutral, because I don't see a better way to address the current issues in a straightforward manner, but if there were one, I'd prefer it.
To be honest I think this motivates a change in the shape of the curve instead (which as far as we know is just arbitrary anyway), so that reasonable behavior doesn't require 30 MB of block data per hour. |
|
My concern is this: Yes, increasing the minimum block size fixes the problem of tx backlog in the short term, but it also potentially reintroduces a problem in that the block limit becomes easier to balloon exponentially if the penalty design is not changed. What I mean is that (regardless of the minimum block size) as the median block size grows, it becomes easier for it to continue growing. At 150kB blocks one normal transaction is ~1% of the block, and so adding another means a 1% increase. As the block size grows, this becomes a 0.5% increase at 300kB, and then a 0.1% increase at 1.5MB blocks, and so on... So similar increases in raw size become cheaper and cheaper, rather than staying constant. Put a different way, as block size increases the current penalty becomes less and less effective because a 1% increase initially represents 1 tx (say 0.01 XMR in fees), then 2 tx, then 4, [...] then 10 or 20 or 100. The fees that offset this rise grow (> 0.1 XMR at 10, > 1 XMR at 100) but the penalty design remains the same. This entire problem could be addressed in an entirely different way if an approach is taken that fixes the penalty scheme rather than (or addition to) the min block size. For example, a scheme could be devised that penalizes based on the absolute block growth, rather than a block growth ratio. In such a scheme, you might have a penalty that assesses 0.005 for a 15kB expansion, 0.015 for a 30kB expansion, 0.035 for a 45kB expansion, etc... This would mean 1 normal tx could be included and still be profitable, but to add a second tx a miner would only include a slightly higher fee paying tx, and then a third would be even higher paying. This is somewhat achieved by the present penalty design, but countered by not using absolute block increases in penalty determination. I guess my whole point here is that this might be a short-term fix, but doesn't really think about the long term, and in my opinion therefore shouldn't just be streamlined in without considering alternative options. I understand the design rationale (average tx sizes increased with RCT, therefore the min block size should likewise increase) but "keep it simple" may not actually be the correct approach here. edit grammar. |
|
Hmm, here's what you get for doing math before eating breakfast, kids.
At 150kB blocks, one normal transaction is ~10% of the block, or a 10% increase to add one additional. At 300kB this becomes 5%, then 1% at 1.5MB, etc... The point stands regardless of my mathematical ineptitude this morning: under the current design, as block sizes increase, it becomes cheaper and cheaper to keep increasing them. Is this really the best design? |
|
Which is the problem we're solving? Let's not invent problems which are not there, please. Nobody is questioning the penalty scheme and the fix is just to make it work as it was working before RCT. Just that.
How is that so? Backlog is not the problem, it would get fixed by itself eventually. But because of the artificial spike in the minimum fee required to make something happen, it will frustrate everyone along the way to 300-ish kB. And it will frustrate everyone again and again whenever we drop to 60kB and have to work our way back to some xx kB where the mempool finally gets emptied, until there's a consistent stream of TX-es sufficient to keep it at >300kB all the time. We've seen it grow to 100kB+ already despite the growing pains. Actually, the formulas never limit the block size. If there's real organic demand it must be allowed to grow otherwise we become Bitcoin. The unnatural barrier still allows them to grow, it will just annoy the hell out of everyone until we get past the 300kB inflexion point. The point of the penalty is not to limit growth, but to throttle the rate at which it can grow and to penalize more the faster it wants to grow. Slow creeping up is ok.
That's not really true. Well it kind of is, but only because the anomaly caused by RCT is in making it more expensive in the 0-1250kB zone, as a direct consequence of RCT TX size! See below the total cost of increase (left is with the current formulas) for a constant market price. I believe it's sufficient to explain everything.
See the difference between RCT and Non-RCT? For Non-RCT, the total cost to increase the block size for 100kB is always the same, if increasing from 100 to 200 or from 1100 to 1200! Thing is, the bigger it is, more TX-es share the cost and it's easier on individual users. A single attacker would always have to bear the cost alone! And that is with a constant market price. If adoption drives the price, then the price grows with the block size and it will actually become more expensive to increase from 1100 to 1200 than it is from 300 to 400. So what you propose is actually already achieved ;) |
|
Just one thing - it CAN'T grow indefinetely without miners agreement. The formulas give them the freedom if they want it but 51% miners can veto any increase as you can't raise the median with only 49% of miner support! Even if it was no-penalty and just the x2 median hard-cap! |
|
So... I had written a pretty comprehensive response starting to outline what I am trying to say about the penalty reward being broken, but it became quite long... too long honestly for a reply in a conversation over a merged PR, especially since the point I want to make starts to get off topic to this specific PR. This is a complete re-write of my original response, which I'm going to post as a new Issue where maybe we can talk about it separate from this specific PR. I fundamentally think that the problem is not the minimum block size but a broken reward penalty calculation, and that probably deserves it's own conversation. So I guess I'll admit defeat in on this particular PR, but I'll respond to your points here because I think there are some things we can still talk about.
In my last reply, I said "I understand the design rationale (average tx sizes increased with RCT, therefore the min block size should likewise increase) but "keep it simple" may not actually be the correct approach here." I acknowledge that this PR is narrow in scope, I just think that in fixing a symptom we're missing a larger problem. My point is that there might be a larger issue here that can and should be addressed at the same time as what this patch is trying to fix, or at the least needs to be discussed in the near future (perhaps after this PR goes live).
While the design makes this true right now, I disagree that it must be allowed to grow no matter what. At some point, Monero will require a second level solution like Lightning to scale to 100s or 1000s of txs per second, if not earlier at 10's of txs per seconds. Either that or we'll need technological breakthroughs or efficiency gains that aren't readily apparent.
The point of the penalty is to prevent miners from spamming the blockchain, growing the blocks to too large of a size, and squeezing out competitors. Miners that wish to attack the blockchain either have to incur costs that would be detrimental to their profits, or they would have to "attack" by growing organically as demand actually occurs, which is not really an attack at all, as you point out.
No, it's definitely true, but probably not 100% relevant here. As median block size grows, any given additional size increase becomes proportionally smaller, and therefore has less of a penalty. 10kB/100kB is bigger than 10kB/500kB, which is bigger than 10kB/1000kB, and so on. Growing at a rate of 1 tx more than the median block size is more expensive with a smaller median than larger. ~300 to ~313 costs more than ~1100 to ~1113 because (1113/1100-1)**2 is smaller than (313/300-1)**2. In terms of percentage growth, yes a 10% growth is the same cost regardless of median block size, but in terms of absolute block growth, cost per kb decreases as block size increases. This may not be ideal.
Ugh, I really dislike when people use winky faces while debating a technical point. That might just be me I guess, as it seems all too prevalent in this space, so I'll forgive and forget. Err... ;) Anyway, let me formulate my full point about the reward penalty formula being less than ideal and I'll write up a new issue in the next couple days that would be a better place to discuss all this. |
I agree that the discussion whether it's broken is off-topic here, but I have now some time to answer so here goes.
I get that. But those statements are really missing something to back them with. If typical TX size was 1kB, we probably wouldn't be having this discussion at all.
Which issue is that? I'd like go back to my point "let's not invent problems" here. If there's an issue, it needs to be presented clearly and with clear arguments as to why it's an issue at all, so we can be on the same page what we're talking about.
I could agree with most of it. But, by "must be allowed to grow" could be read as "51% miners must be free to increase it if they're ready". Why should you or I limit them if they "say" they can handle it? 49% won't cut it so nobody can really "attack". If 51% don't want a median increase it simply won't happen, regardless of the economics of penalty. Why did you skip over this argument? It's a rather strong one, IMO. It's up to the miners what they can handle - why change that?
I'd argue that's not the case. A miner with, say, 20% hashrate can't grow the median if others don't let it grow. Thing is, 50 in 100 blocks must be bigger than X for X to become the new median. Since there's a hard cap at x2 median, the extreme case would be blocks of 60,60,60...120,120,120. If there's 51 in 100 blocks of 60kB, the median stays 60kB. If some bad actor has 51% we have bigger problems, really. Anyway, what's the "original" point was, we will never know as it is in CN authors heads. We can only analyze the consequences of the design and see if it makes sense. To me, it does - and it's quite simple and elegant at that. I'd say it's there to limit the rate and avoid sudden shocks like some miners waking up one day and deciding they want to try a x10. Can't happen, with this. But it could happen over a period of weeks, if there's sufficient user demand (AND 51% miner support) for it otherwise someone would have to pay for it out of his pocket. Again, 51% can veto it at any time and drop it back to minimum and reset the process.
Well, even if you don't lift the block size you can still "spam" up to the cap (we can never know if it's spam or not) and keep it lifted once real usage is gone. That's what the min. do-not-relay fee is there for, to make this cost. So the above is really a weak argument, IMO. Again, it takes 51% to make growth happen.
Maybe I need to clarify better how I got to the "it costs the same", as one chart won't cut it obviously. Let's say legit TX-es account for 100kB and some single actor wants to increase the block size to X. Can he do it? Let's say he keeps broadcasting gazillions of TX-es with the min. fee - sufficient for a desired increase. What happens? 100 -> 110kB - attacker pays for 10kB in fees see where this is going? He pays for all the TX-es he adds, not just the ones which go over the cap! If "legit" ones make some constant amount of kB, the qty of kB he needs to push also grows as the median increases. If he fails for 51 blocks, the size is reset and he must work it back up from 0. How is this not a good solution? Only if he has the hashpower can he avoid paying the fee up to the cap so even the pre-cap fees add up to his costs. But then he's again missing out on revenue from not-his fees, same like he will be missing out on revenue from the penalty. If he's not a miner at all, all those TX-es will get mined by others and they will take his fees. ALL of them. So, the higher we are with the current median, it becomes more kB for the attacker. Yes, the min. fee decreases but the rate of decrease is such that the total price to increase for X kB will be constant - which brings me back tho the chart posted above. We can't just look at one TX. We need to look at what some actor needs to pay in total if he wants to go from size X to size X+Y. For a given Y, the total price Z will be the same, regardless of X. Anyway, if you're still not convinced, we can continue elsewhere as it is out of scope here. |
|
I need to get to bed, but I'd like to reply to clarify where I think miscommunication is muddying this discussion. I will talk more to your points tomorrow, but for now allow me a chance to be more specific in what I am trying to convey, because we clearly are talking about two different things. When I talk about the "expense to increase the median block size", I am not talking about the expense of an "attacker" flooding the network with txs trying to get miners to mine larger block. This is not an attack, and there is no such thing as a non-legit transaction in this case. All transactions mined into a block are legit. Transactions not paying the minimum agreed upon fee will not be propagated by nodes to the network, but they are still legitimate transactions. This is an important point, because while they won't be propagated by network rules, there's nothing in the consensus to say that the are illegal. Therefore, 0 fee transactions can exist, and here's how: by a miner creating a 0 fee transaction and mining it him/herself. Expanding this, any time a block is not full to the minimum block size, a miner can without consequence create however many transactions he or she wants that do not have a fee and include them in his or her next mined block. With no fee, and no penalty. So while an individual must pay a set minimum fee for every transaction they send to the miner network, miners themselves do not have to pay this fee because they can mine the transaction themselves. If their block is not at the median block size, perhaps because there is a temporary lull in txs, they can mine these 0-fee transactions at no cost. If their block is at the median block size, the cost of the tx is equal to the reward penalty. Now, when I talk about the "expense to increase the median block size" I am very narrowly and specifically talking about the expense for a miner or group of miners to increase blocks beyond whatever the median size is at the moment. When I say it is "cheaper" to continue increasing the block size as the block size increases, I am specifically talking about the cost for a miner to grow the block size outward. This could be an attack, or it might not be. That's beside the point. The point is that the "cost" to the miner is the block reward penalty, which is deducted from the base reward + transaction fees. At any median block size, there is an optimal amount for the miner to grow the block size by including extra fee-paying txs. In fact, the miner should always want to grow the block size if the additional transaction fees are greater than the penalty. I would argue that this creates perverse incentives, and a block reward penalty scheme where the penalty is smaller than the increased tx fees of a lowest-fee-paying transaction is not ideal. Finally, and this is a bit of an aside, out of all of this comes another point: the block reward penalty as devised incentivizes miners to make sure that transactions are always at least close to the median size by introducing filler 0-fee transactions at no cost to the miner so that at peak demand times they can rake in the most tx fees possible by including the most fee-paying transactions possible. Once miners increase the median block size to, say, 1000kB, they should objectively want to keep it there because the next time a demand peak occurs, they can much more quickly eat those fees as rewards without having to scale back out the block size, paying block penalties. This cannot possibly be ideal. More tomorrow. Let me know if this clarified any of what I'm trying to say here. |
There aren't just the miners to consider though - someone may wish to run a full node without mining. The current PoW algorithm does make mining more "available" to a wider range of users, but there are still potentially prohibitive upfront costs and varying regional electricity costs to consider. So some segment of users running a full-node without mining could be priced out of their ability to verify all transactions on the network since they never get a "vote". Even miners with modest setups (which are likely to vote for a smaller block) probably have more capital than someone only capable of running a full node. Although perhaps people in this situation should pool their resources ... this is why block size is crazy. Too many game-theory type angles to consider.
A miner can always insert dummy transactions that has a fee - the money is going back to them anyway. Preventing this from occurring seems difficult, if not impossible.
The idea ( @JollyMort correct any mistakes here) is that miners concerned about the larger block would not being do this. However, it seems like the miners most willing to increase the block size are likely the ones with the most hashpower/votes (as they likely have the most access to both CPU and bandwidth to handle it). Hopefully I didn't say anything to absurd, I should read the draft paper for commenting further. |
"Regardless of the economics of the penalty" is not a correct way to look at it because "miners" or "49% of miners" or "51% of miners" are not a unified group unless you consider the case of a 51% attack, which we generally don't. It isn't possible for "51% of miners" to "declde" to do anything. It is the economics as viewed by an individual miner which influence that miner's behavior. A decision of 51% of the miners emerges from that collection of individual decisions (again, assuming no 51% attack). This makes the economics foremost and critical to the matter. The penalty is intended to ensure that any individual miner has a disincentive to increase the block size unless there are sufficient fee-paying transactions, and an incentive to do so if there are. This puts the question of the blocksize largely in the hands of the users (for better or worse; 'users' can include deliberate spammers). For similar reasons, an individual miner stuffing his own blocks up to the median does not make much sense. While there is no penalty, there is still a cost in that those blocks need to transmitted and propagated. Stuffing extra transactions will slow down that process and disadvantage the individual miner, yet merely stuffing that one block does not provide a guarantee (or even necessarily an expectation) that the block size won't fall anyway. Likewise there are additional costs to the individual miner such as processing and storage, again with uncertain benefit. It is a risky play. I've personally thought that maybe there should be a penalty that has non-zoro slope starting from zero and then inflects upward at the median, but it isn't clear whether we will need this. We'll see. |
Since in the described scenario the miner is stuffing the block with his/her own transactions, we can assume negligible processing costs. There's no need to verify as the miner should be able to trust him/herself. Yes, bandwidth is a limiting factor here, but how limiting is it? I did a quick simulation to test this. Here is a python script with adjustable parameters for testing (tested with Python 3.5 but should work with 2.7). This is somewhat back of the envelope, but the assumptions are conservative. The biggest assumptions are
Python Script: import math
# Configurable settings:
Mbit_per_sec = 10
base_reward = 8.2
blocktime = 120.
avg_tx_size_kb = 13
# Shouldn't need to change anything below here, but feel free to check this for errors:
kbps = Mbit_per_sec * 125.
propagation_time_per_kb = 1/kbps
print('================================================================')
print(' Rough estimate of cost of adding extra tx to a block in Monero')
print(' Assumptions:')
print(' 1. Base Reward: {0:.3f} XMR'.format(base_reward))
print(' 2. Connection speed between miners: {0:.1f} Mbps'.format(Mbit_per_sec))
print('================================================================')
for extra_kbs in [50., 100., 200., 400., 600., 800., 1000.]:
n_txs_in_extra_kbs = extra_kbs / avg_tx_size_kb
t = extra_kbs * propagation_time_per_kb
blockchance = (1. - math.exp(-(1/blocktime)*t))
total_cost_of_extra_kbs = base_reward * blockchance
cost_per_extra_kb = total_cost_of_extra_kbs / extra_kbs
cost_per_extra_tx = total_cost_of_extra_kbs / avg_tx_size_kb
print('\nAdding extra {0:4.0f} kB to block:'.format(extra_kbs))
print(' Approx # of extra normal txs: {0:.0f}'.format(n_txs_in_extra_kbs))
print(' Extra block propagation time: {0:.3f} sec'.format(t))
print(' Chance another block found: {0:.3f}%'.format(blockchance*100))
print(' Cost per extra kB included: {0:10.8f} XMR'.format(cost_per_extra_kb))
print(' Cost per extra tx included: {0:10.8f} XMR'.format(cost_per_extra_tx))
print(' Total cost to add extra kBs: {0:10.8f} XMR\n'.format(total_cost_of_extra_kbs))Output: So the cost associated with propagating padded txs is very low, whereby including over 200kBs of extra txs is risking the equivalent of one normal transaction's fee. If you assume better bandwidth speeds (10 Mbps is pretty low), this cost drops even more. Also, mining pools have an extra incentive here: they can utilize the "filler" 0-fee txs to pay miner rewards, meaning that maintaining the median block size slightly above the other tx demand decreases their costs. So I would argue it's not a very risky play at all. Maybe maintaining precipitously larger blocks than necessary would be risky, but maintaining somewhat larger blocks has many advantages to miners. |
I'm pretty clear about all of this in the bodies of my previous posts, and therefore won't rehash. These statements are more of an introductory start to the post, and backed up by the rest of the post.
It seems to me that we are in 100% agreement here... Your second paragraph argued my exact point.
I alluded to this in my reply last night, but the min do-not-relay fee doesn't apply when a miner wants to mine his/her own 0-fee transactions. They have no need to relay the fees. skipping over some stuff to get to here:
Yeah, you're right. Is a new Issue the right place to have a discussion like this? Or perhaps monero-project/meta? |
Yeah, this is true, which is why I didn't argue for some sort of concensus change that removes 0-fee txs. Regardless the fee, if a miner includes it in their own block, they get it back.
Yes, and additionally bigger miners benefit from latency in the network. This has been shown many times over by multiple Bitcoin developers so I won't rehash here. Okay, this should probably be my last post here as we should probably move this discussion to a more visible place. Is a new Issue that place, or perhaps monero-project/meta? There's not really a BIPs equivalent for Monero, so discussions just kind of take place wherever, but what's the ideal place for them? |
ghost
mentioned this pull request
Minimum mixin 4 and enforced ringct is moved from v5 to v6.
v5 is now used for an increased minimum block size (from 60000
to 300000) to cater for larger typical/minimum transaction size.
The fee algorithm is also changed to decrease the base per kB
fee, and add a cheap tier for those transactions which we do
not care if they get delayed (or even included in a block).