New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIP 301: Blind Merged Mining (Consensus layer) #643

Open
wants to merge 20 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@psztorc

psztorc commented Feb 4, 2018

This is a draft of a bip for blind merged mining, the second of two drivechain bips.

psztorc added some commits Feb 4, 2018

@TheBlueMatt

This comment has been minimized.

Contributor

TheBlueMatt commented Feb 4, 2018

@luke-jr luke-jr added the New BIP label Feb 5, 2018

@luke-jr

BIPs are Mediawiki format, not Markdown.

This fails to address backward compatibility.

Furthermore, the spec itself seems strictly inferior to the older merge mining protocol as used by Namecoin etc. The alleged shortcomings this BIP claims the older protocol has, are not in fact true.

Some actual shortcomings in this BIP:

  • Requires changes to the Bitcoin protocol.
  • Limited to merely 256 sidechains.
  • Requires conveying the prevblock for no apparent reason.
  • Requires softforks for sidechains.
  • Includes information on sequence of sidechains in the mainchain for no apparent reason. (Skip count is especially redundant since the prev is already explicit)
  • No merkle tree for sidechain data!
  • Requires full nodes maintain a new database.
  • Requires finding a Bitcoin block for any merged chains.
Specifically, per side:block per side:chain, we track the following 35 bytes of information:
1-byte - ChainIndex (known as "Account Number" in hashrate-escrows.md , or as "Sidechain Number")

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

Where is this file?

This comment has been minimized.

Abstract
==========
Blind Merged Mining (BMM) is a way of mining special extension blocks, ie "sidechains". It produces strong guarantees that the block is valid, for *any* arbitrary set of rules; and yet it does so without requiring miners to actually do any validation on the block whatsoever.

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

Extension blocks and sidechains are two different things.

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

They are indeed. I feel that an asymmetric sidechain is the same as an optional extension block. I will clarify it.

Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain.
Now that we know what our critical data is, and how it is made, how is this data broadcast and stored?

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

It's not made clear why this data is considered critical... It certainly doesn't seem like it should be.

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

I have clarified this part by adding: " This data is "critical" in the sense that it is the minimum amount of data required to define a sidechain. "

4-byte - Message header (0xD3407053)
(n*(32+5))-byte - A sequence of bytes, of the three Mini-Header items (h*, prevBlockRef, ChainIndex).
( We assume that 5 bytes are used for the Critical Data bytes (non h* parts of the Sidechain Mini-Header). For 256 sidechains, a total of 9,478 bytes [1+1+4+256\*(32+5)] are required, conveniently just below the 10 KB scriptPubKey size limit.)

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

Why? This is useless data to the main chain.

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

The mainchain needs this data in order to validate the two styles of "M8" transaction.

This section introduces a new type of transaction, which allows sidechain block creators to request, and pay for, a critical hash to be included in a specific block by mainchain miners. See [the Blind Merged Mining spec](http://www.truthcoin.info/blog/blind-merged-mining/). This txn allows miners to "sell" the act of mining a sidechain block. By taking advantage of this option, miners earn tx fees for mining sidechains...truly "for free". They do not even need to run sidechain nodes, and the tx-fees they earn are in mainchain BTC. As a result, sidechains affect all miners equally and do not affect the mining ecosystem.
M8 will ultimately come in two versions. The second version will be specialized for use in the Lightning Network and must use the full 32-byte prevBlockHash (ironically, this larger transaction is cheaper for the Bitcoin network to process, as it is completely off-chain). The first version of M8, in contrast, cannot be used inside the Lightning Network, but is slightly more space-efficient (using the 2 prevBlockRef bytes to maintain chain order). It is important to make both options available to the user, because some side:nodes may be unwilling or unable to open a payment channels with each main:miner. However, in the long run we expect the lightning version to be preferred.

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

Lightning doesn't require you to have a channel with the recipient...

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

Yes, I have edited this part to instead read "LN connection". Although, it does not change the point I was trying to make.

Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks:
1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.)

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

They can take a merged chain header from someone else risk-free.

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

Is there not a risk that they will get a merged chain header that is ultimately invalid (as I say above)? If so than it is not risk free.

Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks:
1. Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.)
2. Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity).

This comment has been minimized.

@luke-jr

luke-jr Feb 6, 2018

Member

They can accept payment in any way they like... nothing stops them from being paid in bitcoins to include the mentioned merge block header.

This comment has been minimized.

@psztorc

psztorc Feb 10, 2018

The goal here is to make it require zero trust.

The issue is not being paid in BTC (as they will also be paid "in BTC" if they mine a namecoin sidechain [instead of namecoin-altcoin]). Instead, the issue is being paid on the mainchain.

psztorc added some commits Feb 10, 2018

re-reverse the bips
this is what I originally intended, but I forked this branch at the wrong time
move Chris
CS indicated via tweet that he felt he did not contribute enough to be a co-author
@psztorc

This comment has been minimized.

psztorc commented Feb 10, 2018

Furthermore, the spec itself seems strictly inferior to the older merge mining protocol as used by Namecoin etc. The alleged shortcomings this BIP claims the older protocol has, are not in fact true.

These shortcomings are not even issues that I feel strongly about. But they are issues which are constantly raised by Matt Corallo and Peter Todd specifically. They are in Section 4.3 of Blockstream's sidechains whitepaper as well as in this podcast.

I feel that pool operators will just run all of these nodes from a safe place, and then send as little data as possible to the hashers (the physical ASICs). Perhaps you agree with me, but other people don't agree with us!!

Some actual shortcomings in this BIP:

Requires changes to the Bitcoin protocol.

Yes

Limited to merely 256 sidechains.

Of course, we can have sidechains of sidechains, so the real number is unlimited. And we could simply soft fork BMM into the mainchain a second time (for example to add 256 more).

I really do not think there will be more than 256, but I suppose we can't know for sure.

Requires conveying the prevblock for no apparent reason.
Includes information on sequence of sidechains in the mainchain for no apparent reason. (Skip count is especially redundant since the prev is already explicit)

You may have misunderstood the intent here. It is all about creating the conditions for M8, which is the way that sidechain nodes "buy" space in the main:coinbase_txn . To do this, the M8 payment must trigger on something. In fact, we have two possible versions of M8. One triggers on prev, the other on skip count. The ideal M8 payment is M8_v2 a lightning network payment that triggers on prev. But when we need to resort to an on-chain txn then we must use M8_v1. M8_v1, however, unlike the LN payment, can trigger using the index. It actually saves 30 bytes (although the LN txn is better because it uses zero on-chain bytes).

Requires softforks for sidechains.

There are a few reasons why this is a good idea. One is that, if miners do not support a sidechain, it is in danger of having its funds lost, trapped, or stolen. Another is that is not a big deal, because we expect profit-seeking miners to want to add sidechains, as these increase the value of mined BTC as well as miner's total tx-fee revenues.

More important reasons are here: http://www.drivechain.info/faq/#categorical-control

No merkle tree for sidechain data!

All of the data here is mandatory, so a merkle tree would only accomplish two things (as I see it): first, it would waste 32 bytes of space, to accommodate the (pointless) root hash ; and second, it would be a SegWit-like "evil fork" increasing the blocksize limit arbitrarily (or, I suppose, by 32*256 = 8192 bytes).

Requires full nodes maintain a new database.

This is a disadvantage of this proposal, yes. But the database is pretty small, if you ask me.

Requires finding a Bitcoin block for any merged chains.

I feel that the asymmetric model is by far the most practical way to develop sidechains tech. We get half of the problem solved for free, and we can leverage this solved half to make dealing with the unsolved half much easier. For these asymmetric sidechains, nothing is lost by this requirement.

psztorc added some commits Feb 10, 2018

Merge pull request #2 from psztorc/blind-merged-mining
clarifications + backward compatibility
@psztorc

This comment has been minimized.

psztorc commented Feb 10, 2018

I think I am now up to date on Luke's requested changes and clarifications.

m7 op return update
This has been right in the code, but I kept forgetting to update the BIP.
Regular "Merged-Mining" (MM) allows miners to reuse their hashing work to secure other chains (for example, as in Namecoin). However, traditional MM has two drawbacks:
# Miners must run a full node of the other chain. (This is because [while miners can effortlessly create the block] miners will not create a valid payment to themselves, unless the block that they MM is a valid one. Therefore, miners must assemble a *valid* block first, then MM it.)
# Miners are paid on the other chain, not on the regular BTC mainchain. For example, miners who MM Namecoin will earn NMC (and they will need to sell the NMC for BTC, before selling the BTC in order to pay for electricity).

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

I'm not sure these are drawbacks. By eliminating them, you essentially turn all sidechain hashrate into a kind of decentralised Nicehash service. This can be catastrophic, since it essentially sells control to the highest bidder, which can easily be an attacker.

This comment has been minimized.

@psztorc

psztorc May 30, 2018

I'm not sure these are drawbacks.

The first is, I think. Some miners complained about Namecoin's buggy software, and refused to use it -- leading to suboptimal NMC hashrate, and disproportionate 2x-sha256 rewards (which some SC reviewers are strongly against). Also, if/when namecoin software crashed, it (very temporarily) took down the Bitcoin hashing network with it.

The second is really just a convenience. I suppose it is better for miners to "dogfood" (in a sense) their coinbases (they will be more loyal to Namecoin if they are paid in NMC). But in the sidechains case, they are being paid with BTC, and their ASICs have been designed mine BTC.

sells control to the highest bidder

This does give control over block ordering to the highest bidder. I am truly not sure what will happen as a result.

But first of all we are already in a world where control goes to the highest bidder -- today, they must instead bid using illiquid currencies -- ASIC equipment and electrical power. But if they win then they will have "control" (in the same limited sense that miners have control over block ordering today).

But, hashes/second has a time component, whereas the bidding in blind merged mining does not.

On the other hand, when Bitcoin has no block subsidy, an attacker can broadcast a string of "attack transactions" that pay extremely large fees, and thus induce miners to reorg the chain. It really comes down to the wealth generated by the chain, I think (whether it is a main- or side- chain).

=== Sidechain Critical Data ("Sidechain Mini-Header") ===
Specifically, per side:block per side:chain, we track the following 35 bytes of information:

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

This is a huge scalability regression for sidechains, which currently can be increased in number infinitely without adding more than a fixed 32 bytes to Bitcoin's blocks.

This comment has been minimized.

@psztorc

psztorc May 30, 2018

Hmm, yes I think it is.

But it is intended to respond to the allegation that, in practice, miners would need to run a sidechain node. In that case, the scalability regression would be several orders of magnitude greater. Instead of 32 bytes in your example, it would be infinite bytes.

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

The scalability regression affects not just miners, but all full nodes.

This comment has been minimized.

@psztorc

psztorc Jun 2, 2018

That is correct. It is part of the tradeoff required for trustless BMM.

One could still use the technique of BMM, without any code changes to Bitcoin. But it would involve some trust -- likely, side:nodes would have to open (and fund) an account with pools. Side:nodes would then request that the pool find certain side:blocks ("blindly") and the pool would then deduct some value (side:txn_fees - epsilon) from the node's account.

===== Creation =====
By the time Blind Merged Mining can take place, the ChainIndex is globally known (it is the "Account Number" in D1 [see previous BIP], and "nSidechain" in the code). Each sidechain, when activated by soft fork, will take one of the 0-255 available indexes.

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

Why would there only be 256 sidechains?

This comment has been minimized.

@psztorc

psztorc May 30, 2018

We only allocate one byte to identifying a sidechain.

But there can be sidechains-of-sidechains, so in practice there is no real limit.

<img src="bip-blind-merged-mining/bmm-dots-examples.png?raw=true" align="middle"></img>
Since the hashes themselves are already ordered by the mainchain, tracing the blockchain's path by index (prevBlockRef) will be the same as tracing it by identifying a list of hashes. In other words, the ordering given via each side:block's "prevBlockRef" will be isomorphic to an ordering given by each side:block's "prevSideHeaderHash" ... if "prevSideHeaderHash is defined to be the sidechain's equivalent of the mainchain's "prevBlockHash". It will be possible to freely convert from one to the other. See M8 to learn more about how these hashes are requested by sidechain block creators to be included in the mainchain.

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

What about side:blocks that do not meet the difficulty of a valid Bitcoin block (but still remain a valid sidechain block)?

This comment has been minimized.

@psztorc

psztorc May 30, 2018

Such a block would not be a found sidechain block. It would just be a candidate for the next block (perhaps), or else it would be some arbitrary collection of mempool data.

In BMM, a sidechain header meets its difficulty requirement if and only if the header is included in a mainchain Bitcoin block. That is the only way it can meet its "difficulty" requirement. This is a difference from regular merged mining.

==== M7 -- "Blind-Mine the Sidechain(s)" ====
Thus, (for n sidechains) we have a coinbase output with multiple OP_RETURNs (we've changed the tx-standardness policy to allow multiple OP_RETURNs):

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

There is no tx-standardness policy for coinbases (it is a self-contradicting concept)... Nor are such matters of policy subject to BIPs - they are entirely up to implementations and/or individual node operators.

This comment has been minimized.

@psztorc

psztorc May 30, 2018

That may be so, but in practice many people have pointed out to me that I am "not allowed" to use multiple OP Returns. So it definitely interests some readers.

And we did in fact change the standardness policy.

Should we ensure that we have only changed the policy for coinbase txns only?

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

Policy is about off-chain transaction acceptance. There is no policy for on-chain blocks - just consensus rules.

This comment has been minimized.

@psztorc

psztorc May 30, 2018

Ah, I see what you are saying.

I will try to find a way to clarify the sentence.

<pre>
BIP: ????

This comment has been minimized.

@luke-jr

luke-jr May 30, 2018

Member

Let's go with BIP 301.

This comment has been minimized.

@psztorc

@luke-jr luke-jr changed the title from new bip - blind merged mining to BIP 301: Blind Merged Mining (Consensus layer) May 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment