Create 6 Implement dPoW.md

[bidulemachin]

No description provided.

[PQAdvantage]

Thanks for writing that up @bidulemachin. Veriblock is interesting as is the Zencash block delay penalty proposal. I'd also encourage the QRL team to think about if there are ways for the Foundation to establish temporary centralized nodes, mining to increase the hashrate and any other temporary fixes that could offer similar protections in the interim until a new consensus mechanism is adopted that hopefully achieves better security than just standard PoW or PoS.

[hadacnot]

Thanks @bidulemachin . I am kind of convinced dPOW on bitcoin, like Komodo, would not introduce a significant vulnerability to quantum attacks, since hashing on bitcoin is the least quantum weak part on bitcoin. However, I understood that hashing the Komodo chain on bitcoin is done by a small set of yearly elected notary nodes. This is a policy part that needs careful consideration.
Moreover, the signatures that said nodes use on bitcoin are obviously quantum weak, so they could be impersonated by a QC in the future (but again, this should initially be not so crucial because the hardest part would be to create a fake hash of a QRL block, even if a QC takes control of the notary nodes' addresses on bitcoin).

@PQAdvantage I assume the QRL team is already doing something on the lines of what you propose, as hinted at many times on Discord. However I also assume that the test by curefrankosflue shows that said measures were not triggered by a single user buying all Nicehash power.

[cyyber]

Thanks for creating the QIP. But I really don't expect dPoW as a solution. dPoW means the chain needs to rely on another blockchain for its security, and in case of any vulnerability or attack on that chain, will directly effect QRL.

[Ottslayer]

So will the teams path be to implement a PoS algo as soon as possible (like after the last qip was decided) or will there be a shift to a different PoW algo beforehand?

[cyyber]

@Ottslayer Shifting to a different PoW algo just to avoid ASIC is really not a solution. You can see how the Cryptonightv2 was a failure. It failed to effect any ASIC or FPGA device. Although CryptonightR was successful in eliminating ASIC, but that doesnt guarantee how long will it take ASIC or FPGA device to break CryptonightR. Monero is hard forking in every 4 to 6 months, which is not good for network neither for mining pools.

You may consider, Ethereum, they are aware of ASIC mining Ethereum, and they are still working on it. it has been more than 1 year since ASIC devices are mining Ethereum, but they know that hardforking frequently to avoid ASIC, is not a good solution for an Ethereum network. Thus they are working for some permanent solution.

Right now I don't see different PoW algorithm to answer ASIC and hardforking in every few months, will not be a good for a QRL network. So I will rather suggest to use this time to do further research and testing on PoS or PoS derived algorithms.

[Ottslayer]

Thx for the replay cyyber. I am totally with you on that one. I think pointing all resources to some kind of PoS algo is the better way to go from here.
I was just not sure how the teams approach is after the rather huge discussion yesterday.

thanks again, and kudos for all the work you have already done :)

[cyyber]

@Ottslayer thanks for your support. PoS or such derived algorithm are vital for us, but on the other end, its not that simple, like with DPoS if we have got limited number of nodes staking, then if someone take down handful of nodes, the network will stop. Thus we are looking into possible solutions to maintain the balance between speed & decentralisation.

[Ottslayer]

Sure, thats a concern. I guess like during the discord discussion yesterday, the bottom line is to find the right balance between improving the network to make it safer without jeopardizing the network progression towards PoS / go-qrl / smart contracts...

To be honest, I´m quite confident in your capabilities to find the right way to solve this after experiencing the smooth ledger integration. It works like a charm and shows what the team is able to do.

[PQAdvantage]

So I definitely appreciate how much work goes into a HF by @cyyber and the rest of the team, but I strongly believe a HF should happen ASAP with some better 51% protections in light of the results of @curefrankosflue experiment. I see this as an important stopgap measure until the team rolls out a longer term, more secure consensus mechanism (such as a PoW/PoS hybrid that hopefully mitigates the vulnerabilities of one other). The potential solutions I am aware of are as follows:

1. Change the reorg limit to better align with Bittrex confirms to prevent double spends on the primary exchange - This seems like something that should be done at a minimum in the near future even if it is a hassle;
2. Set up as many Foundation-backed mining rigs as feasible to raise the hashrate and increase cost/difficulty of an attack - This also doesn't seem too onerous and doesn't require a HF;
3. Institute a penalty system for delayed block submission such as that adopted by Zencash/Horizen (See https://www.horizen.global/assets/files/A-Penalty-System-for-Delayed-Block-Submission-by-ZenCash.pdf?r=1 and https://www.reddit.com/r/Horizen/comments/9nd2jc/watch_51_attack_mitigation_penalty_system/) - This seems like a great potential solution to me. I admit I'm not fully versed in how it is accomplished, but can help put you in touch with one of the zen leads if you'd like as I know them;
4. Adopt an outsourced solution like Proof of Proof from Veriblock (https://bitcoinmagazine.com/articles/veriblocks-bitcoin-backed-security-protocol-goes-live/) or delayed PoW from Komodo like other small projects have as a stopgap measure while the team works on the long term solution (https://blog.komodoplatform.com/delayed-proof-of-work-explained-9a74250dbb86) - I would not dismiss lightly how much protection piggybacking off BTC provides against a 51% attack, it would be orders of magnitude more difficult to do and is increasingly embraced by a number of other small cap projects like Einsteinium coin, which avoided a 51% attack by an ethical hacker because it was using it. That said, I understand the reluctance to trust another project's solution etc.;
5. Roll out a hybrid PoW/PoS mechanism even if it's not perfect and even if it only allows for Foundation nodes to stake as a stopgap measure while the team works on the long term solution.

Please note, this is not my day job so there may very well be other options out there and I understand QRL team already may employ some of these protections and others that are not disclosed. But if anyone has other ideas, please share! The crypto world moves fast and things have changed a lot since mainnet was launched. I understand the likelihood of attack may be fairly low between now and the regularly scheduled HF, but QRL is finally starting to get some attention in the industry it well deserves and I'd hate for it to suffer a setback right now.

[hadacnot]

Thanks @PQAdvantage for the links. It seems solution 3 has the advantage of not relying on other chains. Is it somehow an evolution of solution 1. By the way, how is the reorg limit implemented presently?

I would avoid solution 5, especially if it involves some masternode from the Foundation, because this would alienate the support of many, given the risk of centralization (we would attract the same kind of criticism that affects IOTA).

[bidulemachin]

@cyyber Yes you are right : a DPoW based on an oher chain will creates dependency between QRL and this other project. Indeed, in case of any vulnerability or attack on that chain, it will directly effect QRL.

But I am talking about BTC ==> as far as I know the bitcoin protocol was never hacked and has the highest hashpower. Do you really think that, in short term, QRL is stronger alone than with BTC ?

The test done by curefrankosflue demonstrates how weak QRL is today on that point. I am not confident on the fact that lowering reorg limit will fix that issue without other side effect or will not create new attack angles.

Please note that I really respect all the hard and good work that has been done by you and the team.

[bidulemachin]

Also, just to clarify one point : I see DPoW as a temporary solution. In long term, I think it is better that QRL will not have any dependency with any other chain.

[surg0r]

@PQAdvantage @hadacnot @bidulemachin @Ottslayer

Securing the QRL network against 51% attack is a primary concern for the team and project. We have considered:

1. reducing the re-org limit (to be below the bittrex confirmation window)
2. a penalty for changing to fork chain from current chain
3. moving to DPOS swiftly by prioritising this over other work flows

We are not particularly in favour of timestamping our blockheader hashes into other chains (DPOW into bitcoin) - that said as a temporising feature it could be implemented. Neither are we in favour of centralised foundation mining or block validation like IOTA.

An issue is that the subject more complex than it first appears. Even reducing the re-org limit introduces a timing attack which can be used to split nodes off from the majority repeatedly and fracture the network into many clusters mining separate distinct chains without convergence onto the longest chain of most work. Currently bitcoin cash is vulnerable to such an attack. Whilst improbable for the QRL as our re-org limit would be >200 blocks it is still worth considering.

We have at length internally discussed disincentivising a late switch to a forked chain (with a fork block greater than x blocks depth from the current blockheight). But such designs fundamentally tinker with the basic principle that all nodes should follow the longest chain of most work.
It is possible to gradually increase such a work-penalty from x blocks depth to y blocks depth before gradually removing the penalty to z blocks depth, such that if left for long enough a forked network would still converge on a single chain of most work. The reticence to implement such a system is that, like a reduced hard re-org limit, it introduces a timing attack to split the network at ~x blocks by a malicious miner.

Perhaps adding a proof-of-work penalty is something we should discuss further?

Our upcoming decentralised on-chain voting QIP (which will be implemented in the upcoming hard fork) will allow decentralised consensus on such design decisions in the future (such as moving to DPOS, changing mining algo etc..).

[PQAdvantage]

@surg0r Thank you for your detailed response, which makes it abundantly clear that you all have been considering this issue in depth from all angles. For me at least, this is probably one of those cases where a little knowledge is dangerous, so I apologize for that. I'm not a dev, but I'm involved enough in the industry to follow the issue. Your comments have helped me appreciate various downsides to the solutions raised, and I'm really getting the sense now that no project has really solved it yet.

I trust the QRL team will try to implement what's most secure until on-chain voting is implemented etc. I'd love to be able to contribute more on the PoW penalty issue, but I'm not there yet, perhaps I'll talk to someone at Zen about it a bit more and report back. I do like the idea of prioritizing work on the consensus mechanism, still personally hoping to see something like a PoS/PoW hybrid as what wins out on that front.

[surg0r]

@PQAdvantage

I would be grateful if you could contribute more on the POW penalty issue because we are always looking for new ideas!

[hadacnot]

Thanks @surg0r ! I am very glad to read you have thoroughly discussed this internally, especially the part on POW penalty.

In the extreme case in which one sets x=1, I understand this would prevent the synchronization of nodes, simply because of internet latency. But is there any metric on the typical standard delay of synchronization between QRL nodes? If one puts x much larger than that, but smaller than (currently Bittrex) exchange confirmation window, would one just increase the risk of fracture in cases of prolonged internet malfunctioning (or government intervention)?

Or I am missing the point?

[cyyber]

@bidulemachin Yes BTC has not yet been hacked, but that doesnt mean QRL should form a dependency over another coin.

@PQAdvantage The idea of using penalty for delay block is something, that could make QRL network stronger. We had a discussion on this before within the team. Moreover, lowering reorg limit is already in our plan for this hard fork.

A penalty system for delayed block submission, is nice idea, although it cannot solve the problem of 51% or timing attack, but atleast it can strengthen the network as the attack will be economically more costly.

@surg0r Before we start discussing about the implementation of penalty system for delayed blocks, it will be worth to decide the re-org limit. The current bittrex confirmation is 350 blocks, so in order to reduce this, we need to reduce our re-org limit lower than 350 blocks. If we keep the re-org limit to 350 blocks to next hard fork, its not going to change the number of confirmations required by bittrex. Once we confirm the re-org limit will be somewhere around X blocks, we should proceed forward about the implementation of the penalty system.

I highly appreciate community involvement into this QIP.

[surg0r]

@hadacnot

I see the issue with setting x=1 is that it is quite natural for two or more blocks to be propagated across the network by different mining pools in the same time period as the network latency for node propagation. Thus the tip of the main chain may have different fork blocks across the network which could take a few blocks for the network to resolve by hash power and converge on an agreed network main chain (tip).

Adding work penalties for the first few blocks deep from the current blockheight therefore will interfere with this natural jostling process amongst miners to fight out the current main chain. I would expect setting x=1 to mean the QRL network would continue to fork for extended periods instead of resolving the forked chain quickly and could even potentially permanently fracture the network into several chains. @cyyber may wish to comment on this?

My personal view would be that such penalties should kick in at a depth back from the tip of the chain to be incompatible with disrupting a simple fork recovery due to temporary internet routing disruption of some of the QRL nodes or competing miners fighting to win a natural fork. I would propose somewhere in the range of 20-100 blocks before such a penalty system activates. Any new chain becoming public that far deep from the current blockheight is probably an attacker. The attacker could still re-org the chain to 20-100 blocks but for a 51% attack to achieve monetary success he would have to exceed the exchange confirmation window (bittrex is ~350 blocks currently) which would be made greatly more difficult with an exponential work penalty between 20-100 to 350 blocks.

We are going to modify a test QRL node to record the depth and frequency of mini-forks and consequent minor natural re-orgs to see objectively what occurs in practice.

[IMac318]

I like the idea of a delayed block submission penalty, but I am curious if it has any real benefit if there is a reorg limit that is less than the exchange confirmation number already in place. Or are people suggesting that the reorg limit is replaced by the delayed submission penalty?

Also, I recall Vitalik proposing a consensus algorithm that makes PoW 99% fault tolerant a while back (paper: https://vitalik.ca/general/2018/08/07/99_fault_tolerant.html). It looks like it could make PoW very secure and also help in an implementation of PoS. Has this been considered at all?

[surg0r]

@IMac318

I like the idea of a delayed block submission penalty, but I am curious if it has any real benefit if there is a reorg limit that is less than the exchange confirmation number already in place. Or are people suggesting that the reorg limit is replaced by the delayed submission penalty?

Well it would potentially allow the exchange to lower the exchange confirmation number substantially if we can show quantitatively that a 51% attack is prohibitively expensive at a certain depth back from the current blockheight. But I think we are discussing the merits of doing both.

Also, I recall Vitalik proposing a consensus algorithm that makes PoW 99% fault tolerant a while back (paper: https://vitalik.ca/general/2018/08/07/99_fault_tolerant.html). It looks like it could make PoW very secure and also help in an implementation of PoS. Has this been considered at all?

I remember reading this when he published it. It has quite a few assumptions and requirements and would move us towards a fairly complex hybrid POW/POS system. Personally my view would be to harden our existing POW implementation with our next hard fork and then work towards pure POS thereafter.

[cyyber]

@hadacnot

x=1 will definitely make the node unstable. Anything between 10 to 20 blocks from where the penalty system actives makes sense. As our block timing is 60 seconds, so 10 to 20 blocks gives atleast 10 to 20 minutes for penalty system to activate which is more than enough for the blocks to propagate among all the nodes.

"But is there any metric on the typical standard delay of synchronization between QRL nodes?"

Talking about metrics, there are several factors by which this delay might get effected, some of them the network speed of the receiver and sender node as well as the distance between those nodes. Other factors also plays a role such as the block size and the traffic in the network such as traffic caused by transactions. Usually QRL block size varies between 257 bytes to 30,720 bytes. Rarely it goes above 30,720 bytes (30 Kb). Under current network condition, for the node which I am running in India, it takes approx 556 ms to receive the latest block.

"If one puts x much larger than that, but smaller than (currently Bittrex) exchange confirmation window, would one just increase the risk of fracture in cases of prolonged internet malfunctioning (or government intervention)?"

QRL current re-org limit is 22,000 blocks, a prolonged internet malfunctioning could even fracture the network if it goes beyond 22,000 blocks. Currently, in such a situation a manual intervention is required and there is no perfect solution for this. Although it is true that with lower re-org limit any such network malfunctioning will increase the risk for the disconnected nodes to make their own forked chain and go beyond the re-org limit which will lead to failure in automatic fork recovery. This issue is common, and its even in many other blockchain, which needs manual intervention to make the node to join the actual chain. However, I don't see this issue on the blockchain running on consensus mechanism like PoS or DPoS.

[surg0r]

So we have searched back through the entire chain from the State of one of our nodes and it seems we have the following re-org fork-branch statistics (they may vary by node):

14470 forks to blockheight 412250
Median length fork-branch is 1, range 1-6 blocks
Typically a fork every ~28 blocks

It is reasonable to say there is no need for a fork greater than 10 blocks to arise under normal network circumstances. Certainly there is no need for a 300 late fork-switch to ever occur and so setting the re-org limit there (below the exchange deposit confirmation limit) makes sense as it easily neuters the benefit of a 51% attack whilst preserving the longest chain of most work behaviour of the network.

We are investigating a timing split attack further but given that is detectable we can modify our standard node implementation to alert that a suspicious network fork has occurred and prevent this vector being economically useful.

[hadacnot]

That's great! Thanks for the statistics. By the way, how do you do that (taking statistics) if you run a node? Is there any guide on how to analyse QRL blockchain?

[cyyber]

The statistics can only be calculated if you are running the node since the genesis block without the node being offline for any moment. As only the nodes which are online since genesis block can be aware of forks. Thus if anyone who runs a node, which needs to sync, will not be able to know about any fork that happened till the block height where it synced as, the other nodes only sends the block on the mainchain during block downloading/syncing process.

Thus we took the state files from one of our node running since genesis, and we wrote a simple python script, to calculate the fork stats.

[hadacnot]

Oh, that should have been obvious indeed...
And, if I understand correctly, the statistics you collected is a good estimate of worldwide forks that your node received.

So, the only thing we are missing here is whether a node, say, in Russia, sent a forked chain to China, that rejected it before it was sent to the node you are collecting statistics on, right? Which means it was shorter than the numbers you mentioned, otherwise it would have been sent also to your node. Is this reasoning correct?

[cyyber]

Well there could be two cases
1> A node broadcasting a forked chain, might have resulted into switching from current mainchain to new forked chain. In such a case, those blocks are broadcasted further to other nodes too and all nodes would have seen that fork.
2> A node broadcasting a forked chain, might not have enough difficulty compared to mainchain, thus the receiving node, will not forward this forked block to other nodes. And so this kind of fork might not been seen by other nodes.

So it is possible, that other nodes, might have seen a different fork compared to the ones that has been seen by our node.

Since QRL nodes have bootstrap ip in config.py , and almost all nodes are connected with it, so the chances are high that our node would have seen most of the forks.

[bidulemachin]

did the team had a look on "Proof of diversity" developped by Nyzo project ?