Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consensus: Proof of Work #3473

Merged
merged 59 commits into from Sep 6, 2019

Conversation

@sorpaas
Copy link
Member

commented Aug 23, 2019

This PR implements PoW consensus engine in Substrate. We currently have a PoW verifier, and a simple thread-based miner. The PoW algorithm and difficulty adjustment algorithm is generic over PowAlgorithm struct. The caller is expected to implement one of this that suits its need.

For mining, we currently just use a simple runtime function PowAlgorithm::mine. It takes a header pre-hash, and is expected to loop until it finds a seal that satisfies the current difficulty.

Things that can be improved, but probably not in this PR:

  • RPC methods for getWork and submitWork.
@sorpaas sorpaas requested a review from 0x7CFE Aug 23, 2019
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
@rphmeier

This comment has been minimized.

Copy link
Member

commented Aug 23, 2019

You can use a pre-runtime digest for the coinbase - that'll put all necessary info in the header. A runtime API to fill it out using the runtime's keystore API would work perfectly for that.

No run-on expressions allowed.
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/primitives/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/primitives/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
sorpaas added 11 commits Aug 26, 2019
The preruntime digest can be intepreted by the runtime as the block author/coinbase.
This makes it more apparent that currently in PoW engine, `difficulty` should
be input, not output.
core/consensus/pow/primitives/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/primitives/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
sorpaas and others added 2 commits Aug 29, 2019
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
@burdges

This comment has been minimized.

Copy link

commented Sep 1, 2019

As an aside, we should make doing parachains with proof-of-stake block production relatively easy and discourage parachains from actually using proof-of-work. It's fine for testing of course but maybe bad optics if we have real parachains doing this.

sorpaas and others added 3 commits Sep 2, 2019
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
/// Algorithm used for proof of work.
pub trait PowAlgorithm<B: BlockT> {
/// Get the next block's difficulty.
fn difficulty(&self, parent: &BlockId<B>) -> Result<Difficulty, String>;

This comment has been minimized.

Copy link
@cloudhead

cloudhead Sep 2, 2019

How would this work for eg. Nakamoto PoW where block difficulty depends on the last 2016 blocks?

This comment has been minimized.

Copy link
@sorpaas

sorpaas Sep 2, 2019

Author Member

The struct that implements PowAlgorithm is expected to take a reference to client Arc<C>. That will give it access to all past blocks, where it can find out their difficulty through auxiliary store. This way we can implement Nakamoto PoW. :)

This comment has been minimized.

Copy link
@cloudhead

cloudhead Sep 2, 2019

Great, thanks!

This comment has been minimized.

Copy link
@0x7CFE

0x7CFE Sep 4, 2019

Member

But what about light clients? Is aux store available for them? I thought they have access to headers only.

This comment has been minimized.

Copy link
@sorpaas

sorpaas Sep 4, 2019

Author Member

As far as I understand light client should have access to aux store. Even if they don't, because we only need to store tiny amount of metadata in aux store, it's also quite easy to move that elsewhere.

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
@DemiMarie-parity DemiMarie-parity self-requested a review Sep 4, 2019
Copy link
Contributor

left a comment

LGTM!

core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
core/consensus/pow/src/lib.rs Show resolved Hide resolved
sorpaas added 2 commits Sep 5, 2019
@0x7CFE
0x7CFE approved these changes Sep 6, 2019
core/consensus/pow/src/lib.rs Outdated Show resolved Hide resolved
1. Make preruntime Optional.
2. Add more docs on what is `preruntie` and `round`.
3. Replace `Default::default` with the approriate type.
@sorpaas sorpaas merged commit b9e1782 into master Sep 6, 2019
9 checks passed
9 checks passed
continuous-integration/gitlab-cargo-check-benches Build stage: test; status: success
Details
continuous-integration/gitlab-cargo-check-subkey Build stage: test; status: success
Details
continuous-integration/gitlab-check-line-width Build stage: test; status: success
Details
continuous-integration/gitlab-check-runtime Build stage: test; status: success
Details
continuous-integration/gitlab-check-web-wasm Build stage: test; status: success
Details
continuous-integration/gitlab-check_warnings Build stage: build; status: success
Details
continuous-integration/gitlab-test-linux-stable Build stage: test; status: success
Details
continuous-integration/gitlab-test-linux-stable-int Build stage: test; status: success
Details
continuous-integration/gitlab-test-srml-staking Build stage: test; status: success
Details
@sorpaas sorpaas deleted the pow branch Sep 6, 2019
@sorpaas sorpaas added A8-looksgood and removed A0-pleasereview labels Sep 6, 2019
@shawntabrizi

This comment has been minimized.

Copy link
Member

commented Sep 6, 2019

@sorpaas Do you have time to work with someone from the docs team to help write some information about this?

How to use it, how it works, etc...

Would be looking to you to answer questions for whomever writes the docs.

@sorpaas

This comment has been minimized.

Copy link
Member Author

commented Sep 7, 2019

@shawntabrizi Yeah for sure!

@JoshOrndorff

This comment has been minimized.

Copy link
Contributor

commented Sep 8, 2019

@sorpaas I've taken the lead on consensus on the docs team. Could we set up a call this week? I'll do my homework first. Tuesday or Wednesday would be fine.

andresilva added a commit that referenced this pull request Sep 17, 2019
* consensus-pow: init primtives and verifier

* consensus-pow: add total difficulty auxiliary

* consensus-pow: implement total difficulty chain selection

* consensus-pow: implement pow import queue

* consensus-pow-primitives: add mine into PowApi

* consensus-pow: implement mining

* Update lock file

* Style fixes

No run-on expressions allowed.

* consensus-pow: refactor register_pow_inherent_data_provider

* consensus-pow: make PowApi::mine yieldable

* consensus-pow: better mining loop

* Add missing license header

* consensus-pow-primitives: clarify the meaning of None for PowApi::verify

* consensus-pow: changing total difficulty addition to use saturating add

* consensus-pow: change mine-loop error to log on error! level

* consensus-pow: allow inserting arbitrary preruntime digest for pow

The preruntime digest can be intepreted by the runtime as the block author/coinbase.

* Fix line width

* More line width fixes

* consensus-pow: separate difficulty, verify API

This makes it more apparent that currently in PoW engine, `difficulty` should
be input, not output.

* srml-pow: implementation of average_span difficulty adjustment

* srml-pow: basic blake2 algo example

* srml-pow-average-span: make it not require genesis config

* srml-pow: add support for authorship

* Missing license headers

* consensus-pow: PowAlgorithm trait generalization

* Missing docs for consensus-pow

* More docs

* node-runtime: bump impl_version

* Add rationale for difficulty type

* consensus-pow: refactor aux_key

* Update lock file

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Remove PowRuntimeAlgorithm

* block_id -> parent_block_id

* Auxiliary data -> auxiliary storage data

* Fix error message

* Fix compile

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix crate description

* More docs

* Address grumbles

1. Make preruntime Optional.
2. Add more docs on what is `preruntie` and `round`.
3. Replace `Default::default` with the approriate type.
DemiMarie-parity added a commit to DemiMarie-parity/substrate that referenced this pull request Sep 17, 2019
* consensus-pow: init primtives and verifier

* consensus-pow: add total difficulty auxiliary

* consensus-pow: implement total difficulty chain selection

* consensus-pow: implement pow import queue

* consensus-pow-primitives: add mine into PowApi

* consensus-pow: implement mining

* Update lock file

* Style fixes

No run-on expressions allowed.

* consensus-pow: refactor register_pow_inherent_data_provider

* consensus-pow: make PowApi::mine yieldable

* consensus-pow: better mining loop

* Add missing license header

* consensus-pow-primitives: clarify the meaning of None for PowApi::verify

* consensus-pow: changing total difficulty addition to use saturating add

* consensus-pow: change mine-loop error to log on error! level

* consensus-pow: allow inserting arbitrary preruntime digest for pow

The preruntime digest can be intepreted by the runtime as the block author/coinbase.

* Fix line width

* More line width fixes

* consensus-pow: separate difficulty, verify API

This makes it more apparent that currently in PoW engine, `difficulty` should
be input, not output.

* srml-pow: implementation of average_span difficulty adjustment

* srml-pow: basic blake2 algo example

* srml-pow-average-span: make it not require genesis config

* srml-pow: add support for authorship

* Missing license headers

* consensus-pow: PowAlgorithm trait generalization

* Missing docs for consensus-pow

* More docs

* node-runtime: bump impl_version

* Add rationale for difficulty type

* consensus-pow: refactor aux_key

* Update lock file

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Remove PowRuntimeAlgorithm

* block_id -> parent_block_id

* Auxiliary data -> auxiliary storage data

* Fix error message

* Fix compile

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/primitives/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix crate description

* More docs

* Address grumbles

1. Make preruntime Optional.
2. Add more docs on what is `preruntie` and `round`.
3. Replace `Default::default` with the approriate type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.