Consensus: Proof of Work #3473

sorpaas · 2019-08-23T18:43:10Z

This PR implements PoW consensus engine in Substrate. We currently have a PoW verifier, and a simple thread-based miner. The PoW algorithm and difficulty adjustment algorithm is generic over PowAlgorithm struct. The caller is expected to implement one of this that suits its need.

For mining, we currently just use a simple runtime function PowAlgorithm::mine. It takes a header pre-hash, and is expected to loop until it finds a seal that satisfies the current difficulty.

Things that can be improved, but probably not in this PR:

RPC methods for getWork and submitWork.

core/consensus/pow/primitives/src/lib.rs

core/consensus/pow/src/lib.rs

core/consensus/pow/primitives/src/lib.rs

rphmeier · 2019-08-23T19:52:47Z

You can use a pre-runtime digest for the coinbase - that'll put all necessary info in the header. A runtime API to fill it out using the runtime's keystore API would work perfectly for that.

No run-on expressions allowed.

core/consensus/pow/src/lib.rs

core/consensus/pow/primitives/src/lib.rs

core/consensus/pow/src/lib.rs

The preruntime digest can be intepreted by the runtime as the block author/coinbase.

… pow

This makes it more apparent that currently in PoW engine, `difficulty` should be input, not output.

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

burdges · 2019-09-01T02:35:30Z

As an aside, we should make doing parachains with proof-of-stake block production relatively easy and discourage parachains from actually using proof-of-work. It's fine for testing of course but maybe bad optics if we have real parachains doing this.

… pow

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

cloudhead · 2019-09-02T10:02:29Z

core/consensus/pow/src/lib.rs

+/// Algorithm used for proof of work.
+pub trait PowAlgorithm<B: BlockT> {
+	/// Get the next block's difficulty.
+	fn difficulty(&self, parent: &BlockId<B>) -> Result<Difficulty, String>;


How would this work for eg. Nakamoto PoW where block difficulty depends on the last 2016 blocks?

The struct that implements PowAlgorithm is expected to take a reference to client Arc<C>. That will give it access to all past blocks, where it can find out their difficulty through auxiliary store. This way we can implement Nakamoto PoW. :)

Great, thanks!

But what about light clients? Is aux store available for them? I thought they have access to headers only.

As far as I understand light client should have access to aux store. Even if they don't, because we only need to store tiny amount of metadata in aux store, it's also quite easy to move that elsewhere.

core/consensus/pow/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

Demi-Marie

LGTM!

core/consensus/pow/src/lib.rs

1. Make preruntime Optional. 2. Add more docs on what is `preruntie` and `round`. 3. Replace `Default::default` with the approriate type.

shawntabrizi · 2019-09-06T15:40:54Z

@sorpaas Do you have time to work with someone from the docs team to help write some information about this?

How to use it, how it works, etc...

Would be looking to you to answer questions for whomever writes the docs.

sorpaas · 2019-09-07T00:28:27Z

@shawntabrizi Yeah for sure!

JoshOrndorff · 2019-09-08T18:28:13Z

@sorpaas I've taken the lead on consensus on the docs team. Could we set up a call this week? I'll do my homework first. Tuesday or Wednesday would be fine.

* consensus-pow: init primtives and verifier * consensus-pow: add total difficulty auxiliary * consensus-pow: implement total difficulty chain selection * consensus-pow: implement pow import queue * consensus-pow-primitives: add mine into PowApi * consensus-pow: implement mining * Update lock file * Style fixes No run-on expressions allowed. * consensus-pow: refactor register_pow_inherent_data_provider * consensus-pow: make PowApi::mine yieldable * consensus-pow: better mining loop * Add missing license header * consensus-pow-primitives: clarify the meaning of None for PowApi::verify * consensus-pow: changing total difficulty addition to use saturating add * consensus-pow: change mine-loop error to log on error! level * consensus-pow: allow inserting arbitrary preruntime digest for pow The preruntime digest can be intepreted by the runtime as the block author/coinbase. * Fix line width * More line width fixes * consensus-pow: separate difficulty, verify API This makes it more apparent that currently in PoW engine, `difficulty` should be input, not output. * srml-pow: implementation of average_span difficulty adjustment * srml-pow: basic blake2 algo example * srml-pow-average-span: make it not require genesis config * srml-pow: add support for authorship * Missing license headers * consensus-pow: PowAlgorithm trait generalization * Missing docs for consensus-pow * More docs * node-runtime: bump impl_version * Add rationale for difficulty type * consensus-pow: refactor aux_key * Update lock file * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/primitives/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Update core/consensus/pow/primitives/src/lib.rs Co-Authored-By: Sergei Pepyakin <sergei@parity.io> * Remove PowRuntimeAlgorithm * block_id -> parent_block_id * Auxiliary data -> auxiliary storage data * Fix error message * Fix compile * Update core/consensus/pow/primitives/src/lib.rs Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com> * Update core/consensus/pow/src/lib.rs Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com> * Update core/consensus/pow/primitives/src/lib.rs Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com> * Update core/consensus/pow/src/lib.rs Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com> * Fix crate description * More docs * Address grumbles 1. Make preruntime Optional. 2. Add more docs on what is `preruntie` and `round`. 3. Replace `Default::default` with the approriate type.

IPSE-TEAM · 2019-11-06T15:04:10Z

This PR implements PoW consensus engine in Substrate. We currently have a PoW verifier, and a simple thread-based miner. The PoW algorithm and difficulty adjustment algorithm is generic over PowAlgorithm struct. The caller is expected to implement one of this that suits its need.

For mining, we currently just use a simple runtime function PowAlgorithm::mine. It takes a header pre-hash, and is expected to loop until it finds a seal that satisfies the current difficulty.

Things that can be improved, but probably not in this PR:

RPC methods for getWork and submitWork.

come on, How to implement this RPC?

sorpaas added 6 commits August 23, 2019 15:59

consensus-pow: init primtives and verifier

ea409c6

consensus-pow: add total difficulty auxiliary

ceb0595

consensus-pow: implement total difficulty chain selection

634e789

consensus-pow: implement pow import queue

19c215c

consensus-pow-primitives: add mine into PowApi

ba0d5c7

consensus-pow: implement mining

6d78b8c

sorpaas added the A3-in_progress Pull request is in progress. No review needed at this stage. label Aug 23, 2019

sorpaas requested a review from 0x7CFE August 23, 2019 18:43

Update lock file

ac6daab

rphmeier reviewed Aug 23, 2019

View reviewed changes

core/consensus/pow/primitives/src/lib.rs Show resolved Hide resolved

rphmeier reviewed Aug 23, 2019

View reviewed changes

core/consensus/pow/primitives/src/lib.rs Show resolved Hide resolved

rphmeier reviewed Aug 23, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved

rphmeier reviewed Aug 23, 2019

View reviewed changes

core/consensus/pow/primitives/src/lib.rs Outdated Show resolved Hide resolved

Style fixes

a94a48f

No run-on expressions allowed.

tomaka reviewed Aug 26, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved

consensus-pow: refactor register_pow_inherent_data_provider

286f260

pepyakin reviewed Aug 26, 2019

View reviewed changes

tomaka reviewed Aug 26, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved

sorpaas added 11 commits August 26, 2019 19:31

consensus-pow: make PowApi::mine yieldable

41d59d3

consensus-pow: better mining loop

766426f

Add missing license header

a434955

consensus-pow-primitives: clarify the meaning of None for PowApi::verify

69b0fc1

consensus-pow: changing total difficulty addition to use saturating add

573c39f

consensus-pow: change mine-loop error to log on error! level

dd85dae

consensus-pow: allow inserting arbitrary preruntime digest for pow

67bda93

The preruntime digest can be intepreted by the runtime as the block author/coinbase.

Fix line width

90e7639

More line width fixes

ccb1445

Merge branch 'master' of https://github.com/paritytech/substrate into…

95d75d7

… pow

consensus-pow: separate difficulty, verify API

0198ae5

This makes it more apparent that currently in PoW engine, `difficulty` should be input, not output.

Update core/consensus/pow/src/lib.rs

5ad5755

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

sorpaas and others added 3 commits September 2, 2019 11:43

Merge branch 'master' of https://github.com/paritytech/substrate into…

1a6cc58

… pow

Update core/consensus/pow/primitives/src/lib.rs

20b1ab1

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

Merge branch 'pow' of https://github.com/paritytech/substrate into pow

68316b6

cloudhead reviewed Sep 2, 2019

View reviewed changes

Demi-Marie reviewed Sep 4, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved

Update core/consensus/pow/src/lib.rs

bf1576b

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

Demi-Marie self-requested a review September 4, 2019 16:24

Demi-Marie approved these changes Sep 4, 2019

View reviewed changes

0x7CFE suggested changes Sep 5, 2019

View reviewed changes

sorpaas added 2 commits September 5, 2019 16:26

Fix crate description

910f173

More docs

4732737

0x7CFE approved these changes Sep 6, 2019

View reviewed changes

0x7CFE reviewed Sep 6, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved

0x7CFE reviewed Sep 6, 2019

View reviewed changes

core/consensus/pow/src/lib.rs Show resolved Hide resolved

Address grumbles

b227b98

1. Make preruntime Optional. 2. Add more docs on what is `preruntie` and `round`. 3. Replace `Default::default` with the approriate type.

sorpaas merged commit b9e1782 into master Sep 6, 2019

sorpaas deleted the pow branch September 6, 2019 13:45

sorpaas added A8-looksgood and removed A0-please_review Pull request needs code review. labels Sep 6, 2019

gnunicorn mentioned this pull request Oct 5, 2019

[Consensus] Implement Grin's Cuckoo Cycle proof of work on substrate substrate-developer-hub/hacktoberfest#41

Open

2 tasks

sorpaas mentioned this pull request Dec 4, 2019

substrate-casper: Safety of consensus and runtime upgrade interaction paritytech/shasper#189

Open

sorpaas mentioned this pull request Oct 24, 2022

Membership request polkadot-fellows/seeding#50

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consensus: Proof of Work #3473

Consensus: Proof of Work #3473

sorpaas commented Aug 23, 2019 •

edited

rphmeier commented Aug 23, 2019 •

edited

burdges commented Sep 1, 2019

cloudhead Sep 2, 2019

sorpaas Sep 2, 2019

cloudhead Sep 2, 2019

0x7CFE Sep 4, 2019 •

edited

sorpaas Sep 4, 2019

Demi-Marie left a comment

shawntabrizi commented Sep 6, 2019

sorpaas commented Sep 7, 2019

JoshOrndorff commented Sep 8, 2019 •

edited

IPSE-TEAM commented Nov 6, 2019

Consensus: Proof of Work #3473

Consensus: Proof of Work #3473

Conversation

sorpaas commented Aug 23, 2019 • edited

rphmeier commented Aug 23, 2019 • edited

burdges commented Sep 1, 2019

cloudhead Sep 2, 2019

Choose a reason for hiding this comment

sorpaas Sep 2, 2019

Choose a reason for hiding this comment

cloudhead Sep 2, 2019

Choose a reason for hiding this comment

0x7CFE Sep 4, 2019 • edited

Choose a reason for hiding this comment

sorpaas Sep 4, 2019

Choose a reason for hiding this comment

Demi-Marie left a comment

Choose a reason for hiding this comment

shawntabrizi commented Sep 6, 2019

sorpaas commented Sep 7, 2019

JoshOrndorff commented Sep 8, 2019 • edited

IPSE-TEAM commented Nov 6, 2019

sorpaas commented Aug 23, 2019 •

edited

rphmeier commented Aug 23, 2019 •

edited

0x7CFE Sep 4, 2019 •

edited

JoshOrndorff commented Sep 8, 2019 •

edited