Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement sharechain for namecoin merged mining #265

Closed
squidicuzz opened this issue Aug 19, 2015 · 11 comments

Comments

@squidicuzz
Copy link

commented Aug 19, 2015

Currently merged mining namecoin is the equivalent of solo-mining, there is no sharechain to track the work of miners. Previous discussions about the issue. Lets work on and implement a merged mining sharechain to pool miners work for namecoin.

I spoke with forrestv about some ideas on how to accomplish this. Gist of chat, drazisil and I have been working on a branch and exploring some ideas. So far we have done some basic testing with creating a p2pool for namecoin.
Help is welcome, we are usually available on irc. Will use this ticket to track and post updates if we are able get anywhere.

@JeremyRand

This comment has been minimized.

Copy link

commented Aug 19, 2015

This would be very nice to have if it's possible, and would potentially qualify for a bounty from the Namecoin devs (I'd be willing to put in $50 USD). We'd love to see more decentralized mining, and a sharechain for merge-mined coins seems like a good way to accomplish this. If assistance from Namecoin devs is needed for any of this, let us know.

@forrestv

This comment has been minimized.

Copy link
Member

commented Aug 19, 2015

The simplest, cleanest, and potentially only possible (definitely not easiest, though) solution would be for Namecoin to adopt MM2 (Merged Mining v2, what P2Pool uses, as it's a sidechain too, see https://github.com/forrestv/mm2-spec for an incomplete spec). MM2 reduces the size of auxiliary PoWs to a small, constant size by hashing most of the parent chain's generation transaction into an SHA256 midstate instead of storing the entire literal parent chain's generation transaction.

P2Pooled mining of merge mined chains with MM1 would require each P2Pool share to include the necessary auxiliary PoW proof, which would require prohibitive bandwidth, as each share would need to include the current Bitcoin-P2Pool generation transaction (currently ~10kB, but could be arbitrarily large), which would increase the size of shares by a factor of 4 or more (currently P2Pool shares are ~3kB).

With MM2, P2Pool shares would only need to carry an MM2 auxiliary proof, which is less than 500 bytes. Namecoin itself would also benefit somewhat, as its future bandwidth and storage requirements would decrease slightly, and no longer be affected by the size of the parent chain's generation transactions.

Of course, this requires a Namecoin hardfork, and so might be completely impossible. If there's any hope for this idea, I can finalize the MM2 spec and possibly write a reference implementation for Namecoin.

@JeremyRand

This comment has been minimized.

Copy link

commented Aug 19, 2015

@forrestv That is an interesting proposal. We're theoretically open to the idea of replacing the merged mining mechanism with something that is better engineered, depending on what the advantages are. We've discussed merged mining with Luke-Jr (he's working on the merged mining spec for Blockstream's sidechains) -- his spec also makes the auxpow header fixed-size, though the way he does it is by making Bitcoin an auxpow chain. Given that it's unclear to us whether Bitcoin will hardfork in favor of Blockstream's spec, we're kind of taking a wait-and-see approach before we make commitments on what we're going to do there. We were unaware that P2Pool had a merged mining spec that also aims to improve on what Namecoin does -- thank you for bringing this to our attention.

If we replace Namecoin's merged mining system, my guess is that we would want to carefully compare our options, which I guess would include both P2Pool's spec and Blockstream's spec, depending on whether Bitcoin accepts Blockstream's spec (which won't be certain for a while). Pinging our lead C++ developer @domob1812 to see what he thinks about this.

@domob1812

This comment has been minimized.

Copy link

commented Aug 19, 2015

I think it would be a very good thing to make auxpow size constant and predictable for Namecoin. We're definitely willing to hardfork for this, especially since there are other ideas for a fork as well that could be combined. However, we also need to gauge miner support (see below).

I don't fully understand the proposal (it seems to be not fully fledged out, or maybe I just don't grasp the implied details), though. It specifies that the auxpow root hash is moved to an OP_RETURN output of the coinbase tx instead of the coinbase input scriptSig, but it does not specify how that allows to construct an auxpow of constant size. Would you mind giving me more information about how that works? I. e., how would such an auxpow look like? I presume that the "SHA256 midstate hash" you mentioned above is enabled by moving the root hash to the very end of the coinbase tx. Is that correct?

Regarding miner support, if I understood the proposal correctly, it means that a miner has to update their merge-mining setup to place the root hash in a different position as before. If they do not only merge-mine Namecoin but other coins as well, it even means that they may need to put both a root hash (for other coins) as before in the scriptSig and the new root hash for Namecoin in the last output. We definitely have to ask them first whether this would be ok or not (I guess, in particular, F2Pool).

Finally, another thing not directly related to this proposal but merge-mining in general: I think it would be cool to get rid of "abusing" the nVersion of Namecoin block headers to carry merge-mining information (like the chain ID and auxpow flag). This may come "soon" in conflict with Bitcoin efforts to use nVersion more thoroughly for things like fork voting. We could, instead, place all this "merge-mining meta data" just after the initial 80 bytes of the block header; this needs also a hard fork but I see no other issues with it. (In particular, no changes to hashing of the header are made. The auxpow data does not need to be authenticated by PoW, since it is just part of the proof itself and all critical data is in the initial 80 bytes.) This could be combined with a potential upgrade of merge-mining in another way.

@domob1812

This comment has been minimized.

Copy link

commented Aug 19, 2015

For the record: I think, however, that Blockstream's proposal of a better format is much nicer and cleaner. I would prefer to use that if there is any reasonable chance for it to be available within, say, the next year or so.

@drazisil

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2015

@forrestv I'm love to see the MM2 spec, (or point me at the code chunks). How much work would it be to create a plugin that converts MM1 from the namecoind (or others) to MM2 prior to shipping it out to the sharecain?

@drazisil

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2015

@domob1812 Could you point me to what exists of Blocktream's MM system, if there are any docs?

@forrestv

This comment has been minimized.

Copy link
Member

commented Aug 19, 2015

I don't fully understand the proposal (it seems to be not fully fledged out, or maybe I just don't grasp the implied details), though. It specifies that the auxpow root hash is moved to an OP_RETURN output of the coinbase tx instead of the coinbase input scriptSig, but it does not specify how that allows to construct an auxpow of constant size. Would you mind giving me more information about how that works? I. e., how would such an auxpow look like? I presume that the "SHA256 midstate hash" you mentioned above is enabled by moving the root hash to the very end of the coinbase tx. Is that correct?

Sorry about not having it more complete before showing it to others. I added a pseudocode example of how a MM2 proof is verified, which also conveys what information a MM2 proof consists of. I intend to continue to expand the spec over the next day. To answer your specific questions:

It is not truly constant size, but O(log (number of transactions in parent chain block) + log (number of MM2 sidechains being mined)), but practically constant size, with a practical upper bound of around 1kB. See the pseudocode for exactly what the proof is composed of, but generally, it establishes a provable link from the hash of whatever data you want to the hash of the parent block, which matches a given target. It does this using "Merkle links" (AKA Merkle branches) and "hash links" (SHA-256 midstate and small amount of other necessary data). It goes:

  1. From the hash of your auxiliary data
  2. To the Merkle root of a tree of all the MM2 sidechains' auxiliary data (via a Merkle link)
  3. To the hash of the parent chain's generation transaction (via a hash link)
  4. To the Merkle root of the parent chain's transactions (via a Merkle link)
  5. To the hash of the parent chain's block (by inserting the calculated Merkle root into the included block header)

Just as you can prove that a Merkle tree with a given root hash contains a certain leaf node, you can prove that a block of data with a given hash ends with some certain data by using the SHA-256 midstate of all the prior data. So yes, you're correct - the MM2 Merkle root is placed at the end of the generation transaction to enable efficient use of the midstate/hash link idea.

@phelixbtc

This comment has been minimized.

Copy link

commented Aug 19, 2015

As mentioned before on bitcointalk NMDF would certainly pitch in half a Bitcoin or so.

@phelixbtc

This comment has been minimized.

Copy link

commented Dec 28, 2015

@forrestv We are currently thinking about a hard fork and MM2 again: https://forum.namecoin.info/viewtopic.php?f=5&t=2466 You are more than welcome to chime in.

@halvors

This comment has been minimized.

Copy link
Contributor

commented Jan 28, 2016

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.