This commit provides a functional implementation of graphene blocks. All unit tests pass and the QA test grapheneblocks.py also passes. From log output it is evident that graphene blocks are requested, created, serialized, deserialized, and reconstructed. However there is still work to be done. My general approach has been to largely replicate the workflow of xthin blocks. Thus I have replicated large portions of the xthin code. It's not clear that this is preferable to sharing code between the two block types. There also remains the question of transaction ordering for graphene blocks. If it is possible to commit a canonical ordering before graphene, then we will want to change this patch accordingly.
The code will also require further optimization (primarily block size optimization) before being deployed to production. My aim is to initiate a review for the basic workflow now and continue to work on optimization while graphene is running on the testnet. The following optimization tasks remain.
Hey Andrea, my pleasure to contribute! Thanks for taking a look.…
On Fri, Feb 16, 2018 at 10:01 AM, Andrea Suisani ***@***.***> wrote: Hi @bissias <https://github.com/bissias>, I'm eager and excited to test and review the code you submitted. Just want to thank you in advance for the work you have done here! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#973 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA9WOOXUixjBnefSHReunZRpqNZDHcA2ks5tVZg2gaJpZM4SIYHy> .
On Fri, Feb 16, 2018 at 4:06 PM, bissias ***@***.***> wrote: Hey Andrea, my pleasure to contribute! Thanks for taking a look. Just a word of warning, I'm new to the Bitcoin code and c++, so expect it to be a little rough ;-)
don't worry I consider myself c++ newbe. BTW, I'm on the BU slack channel (username "george"). Are you on as well?
yes I'm on BU slack using the same nick I'm using here…
On Fri, Feb 16, 2018 at 10:01 AM, Andrea Suisani ***@***.***> > wrote: > Hi @bissias <https://github.com/bissias>, > I'm eager and excited to test and review the code you submitted. Just want > to thank you in advance for the work you have done here! > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#973# issuecomment-366258901>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ AA9WOOXUixjBnefSHReunZRpqNZDHcA2ks5tVZg2gaJpZM4SIYHy> > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#973 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZrE847f5NUzxXm8iku_syosa6wXkMuks5tVZlygaJpZM4SIYHy> .
the travis integration test failure are due to source code formatting problem, they are not actual failures. see https://travis-ci.org/BitcoinUnlimited/BitcoinUnlimited/jobs/342353084#L698 for more details
to get your files automatically formatted add this snippet to your
and install clang-format-3.8 on you deev machine
(I'm supposing you are using a relatively recent ubuntu linux distro for your dev env)
Ok will do. Do you want me to just start a new pull request after making these formatting changes?…
On Fri, Feb 16, 2018 at 10:32 AM, Andrea Suisani ***@***.***> wrote: the travis integration test failure are due to source code formatting problem, they are not actual failures. see https://travis-ci.org/ BitcoinUnlimited/BitcoinUnlimited/jobs/342353084#L698 for more details to get your files automatically formatted add this snippet to your .git/config file in your cloned repo: [filter "bitcoin-clang-format"] clean = "contrib/devtools/clang-format.py format-stdout-if-wanted clang-format-3.8 %f" smudge = cat and install clang-format-3.8 on you deev machine (I'm supposing you are using a relatively recent ubuntu linux distro for your dev env) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#973 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA9WONCs43_h0q3okRjWCa_v8BYa4pvdks5tVZ91gaJpZM4SIYHy> .
just fix the formatting problems, committing the changes and push it. Github will automatically update the PR for you w/o loosing all the comments and the reviewed already made.
if you want before pushing you can squash the commit containing the formatting fix to the previous one
@bissias : Alright, awesome!
Funny coincidence, but just a couple hours ago, I asked Gavin Andresen on the BU slack when and where (which code fork) implementations of the Graphene protocol are to be expected. He told me to ask Brian Levine - and before I could do that, here we are :) I assume you work in his group?
I am working on preconsensus in the form of weak blocks, as described by Peter Rizun in his paper.
One of the things that I want to do, but which would be kind-of overlapping with your work is transmitting weak blocks as delta blocks. What I do right now is to hijack @ptschip's
But yeah, so in terms of actual transmission of my deltablocks I really still need to do something proper :)
But I imagine that it should be very doable to also use the graphene protocol or a variant that would point to a weak block as the basis and then just transmit the delta that is not in the weak block?
If so, I like to take the opportunity to remark here that it would be great if graphene could provide such a hook and provisions to use it in a "deltablocks" scheme as well. That way, both could be combined in the longer run and even further reduce the burstiness of block transmission.
I guess I should also remark that I still consider the whole notion of weakblocks as experimental, and even though I am optimistic that this will deliver benefits, it is in experimental stages still. The proof for that is in the pudding.
What do you think about a delta scheme for graphene?
@awemany, that is a funny coincidence! Glad to finally have something to submit for discussion. And yes I'm working in Brian's group.
I'm only familiar with the basic idea of weak blocks, but my understanding is that you need data structures for both the weak and final blocks. The final block will point to a weak block and may additionally contain some new transactions.
From what you wrote, it sounds like you're considering using something like graphene for the final block? It seems reasonable in that final and graphene blocks share the same objective of communicating some set of transactions that are probably already on the recipients machine. But what hooks did you have in mind exactly?
The core logic for graphene that performs set reconciliation is pretty small. It's the small details of the overall protocol that constitute the majority of the code. So one way I could enable reuse of the graphene code would be to factor out that core logic into a class that could be used outside of the graphene protocol.
Kind of! Weak blocks is what Peter Rizun's called subchains in his paper: https://www.bitcoinunlimited.info/resources/subchains.pdf
The idea of weak blocks is to basically publish partial solutions of finding a block with reduced difficulty (but while searching for a strong block, of course, so the miners essentially do the same thing as before!). As it is found, it will be sent around and kept as an increment to the state of the chain by all participating nodes ("deltablocks"). It is meant as a non-sybil-able preconsensus scheme of what goes into the next strong block. It is assumed that participating nodes will refer to the last weak block when building their next block and enjoy easier propagation of their solutions as most of the block's entropy can be referred to by pointing to the weak block as a 'delta' (the incentive to use it). A side benefit is faster "fractional confirmations" on top of the 0-conf of transactions.
Now, the requirement is added that a node participating in weakblock propagation will build its next block (weak or strong) on top of the last weak block (or rather: longest chain), in the following way (currently; this should of course be adapted for things like a preferred ordering etc.):
Assuming full propagation of such a weakblock, this means that it can then send out any found solution (strong or weak) by referring to the weakblock it is based upon, plus just the set of transactions on top of that.
In terms of graphene, if I understand the approach correctly, this would mean it looks like the mempool is reduced to just those txn that are not in the previous weak block and a single extra hash would be necessary in a "deltagraphene" message that refers to this last propagated weak block.
And this is what I am wondering about: How to make it so that one can say "these transactions are for sure in the transmitted block, see hash so and so here" and thus further reduce the amount of transactions that the graphene set reconciliation has to deal with: By a factor of weakblockrate / strongblockrate assuming no orphaning problems. Basically, what I like to have is a way to say "transmit this CBlock using graphene, but tell graphene that these transactions from that other block (except CB) are for sure in this block, and in the same order".
The advantage of weakblocks over just reducing the block time interval is that this purely an opt-in scheme that can also be gradually implemented by participating miners and does not need a chain fork nor a change of this fundamental variable and all the assumptions that have been built on top of it.
All other code stays the same and code that is ignorant of weak blocks will see the chain progress just like it did before.