Aura and Slots refactoring #8386

bkchr · 2021-03-17T13:52:20Z

Some further small refactoring of sc-consensus-slots and sc-consensus-aura:

SlotData::slot_duration now returns a std::time::Duration to be more abstract over the implicit "u64 means milliseconds" requirement ;)
Add build_aura_worker function to only create an AuraWorker that can be run by the caller.

polkadot companion: paritytech/polkadot#2634

paritytech/substrate#8386

bkchr · 2021-03-17T14:41:39Z

(This pr is not audited, but also does not require an audit. However, the COMPUTER wants to have a label ;))

andresilva · 2021-03-17T16:26:28Z

client/consensus/aura/src/lib.rs

@@ -75,7 +75,7 @@ pub use sc_consensus_slots::SlotProportion;
 type AuthorityId<P> = <P as Pair>::Public;

 /// Slot duration type for Aura.
-pub type SlotDuration = sc_consensus_slots::SlotDuration<u64>;
+pub type SlotDuration = sc_consensus_slots::SlotDuration<sp_consensus_aura::SlotDuration>;


Beautiful 😂

Yeah I know :P Some more future refactoring :P

andresilva · 2021-03-17T16:38:39Z

primitives/timestamp/src/lib.rs

@@ -100,8 +106,8 @@ impl From<Timestamp> for u64 {
 	}
 }

-impl From<sp_std::time::Duration> for Timestamp {
-	fn from(duration: sp_std::time::Duration) -> Self {
+impl From<Duration> for Timestamp {


We can maybe remove this one if it's unused.

paritytech/substrate#8386

octol · 2021-03-17T20:57:34Z

client/consensus/aura/src/lib.rs

@@ -596,10 +666,10 @@ mod tests {
 					let inherent_data_providers = InherentDataProviders::new();
 					register_aura_inherent_data_provider(
 						&inherent_data_providers,
-						slot_duration.get()
+						slot_duration.slot_duration()


Gives the code a little Java flavour ;)

andresilva · 2021-03-17T21:22:40Z

bot merge

ghost · 2021-03-17T21:22:42Z

Waiting for commit status.

ghost · 2021-03-17T21:39:17Z

Merge failed: "Required status check \"continuous-integration/gitlab-check-polkadot-companion-status\" is expected."

bkchr · 2021-03-17T23:25:55Z

bot merge

ghost · 2021-03-17T23:25:57Z

Trying merge.

* Companion for Substrate#8386 paritytech/substrate#8386 * "Update Substrate" Co-authored-by: parity-processbot <>

@andronik

* ci: initial fuzzer job * erasure-coding: update fuzzer Cargo.lock * syntax fix * try this first * install honggfuzz deps * try not * try if else * try SIGINT * ignore hfuzz dirs * ??? * bash is growing on me * decouple builds from running * fix a typo * try copying dirs * fix indentation * try using absolute paths * another try * caching is not worth it * remove outdated needs * cleanup and add futher TODOs * Update .github/workflows/honggfuzz.yml * more diagnostic logs for approval-voting (#2618) * Backing and collator protocol traces including para-id (#2620) * improve backing/provisioner spans * span for collation requests * add para_id to unbacked candidate spans * differentiate validation-construction and find-assignment in selection * better find-assignment spans * organize unbacked-candidate spans directly under job root * Update node/core/provisioner/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software> * add tracing when no assignment in candidate selection (#2623) * New slots/auctions architecture (#2294) * TODOs * Add auctions.rs, comment on changes needed. * Remove cruft from slots * Remove more from auctions.rs * More logic drafting in slots. * More logic in slots.rs * patch some errors * more fixes * last nit * Cleanups in slots.rs * Cleanups in slots.rs * patches * make build * crowdloan to new api * auction compile * Use ParaId instead of FundIndex in Crowdloan (#2303) * use paraid instead of fundindex * Update crowdloan.rs * check caller is manager * Auction tests and fix build warnings. * Configurable origin for initiating auctions * Remove on_finalize * #2303 (manual merge) * Tests for Slots * some registrar tests * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update runtime/common/src/slots.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Slots uses Registrar for CurrentChains * swap works test * on swap impl * traitify parachain cleanup * explicit lifecycle tracking for paras * initial implementation of lifecycles and upgrades * clean up a bit * Update runtime/common/src/slots.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * fix doc comment * more rigid lifecycle checks * include paras which are transitioning, and lifecycle query * format guide * update api * update guide * explicit outgoing state, fix genesis * handle outgoing with transitioning paras * Revert "explicit lifecycle tracking for paras" This reverts commit 4177af7. * remove lifecycle tracking from registrar * do not include transitioning paras in identifier * Update paras_registrar.rs * final patches to registrar * Fix test * use noop in test * clean up pending swap on deregistration * finish registrar tests * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Apply suggestions from code review * Use matches macro * Correct terms * Apply suggestions from code review * Remove direct need for Slots and Registrar from Crowdloan * Rejig things slightly * actions queue * Revert "actions queue" This reverts commit b2e9011. * Traitify Auction interface. * Mockups and initial code for Crowdloan testing * One test... * collapse onboarding state * fix some crowdloan tests * one more * start benchmarks for auctions * benchmark bid * fix more crowdloan tests * onboard and begin retirement no longer exist * Revert "onboard and begin retirement no longer exist" This reverts commit 2e100fd. * Simplify crowdloan and make it work. * Fixes * fix some * finish merge fixes * fix refund bug in auctions * Add traits to Registrar for tests and benchmarks * fix more auction benchmarks * Fix TestAuctioneer * finish crowdloan benchmarks * start setting up full integration tests * expand integration tests * finish basic integration test * add more integration tests * begin slots benchmarks * start paras registrar benchmarks * fix merge * fix tests * clean up paras registrar * remove println * remove outdated cleanup config * update benchmarks * Add WeightInfo * enable runtime-benchmarks feature flag * complete swap benchmark * add parachains and onboarding into westend * add benchmarks and genesis * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * fix benchmark execution * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * Use `new_raise_len` in crowdloan on_initialize * Update paras_registrar.rs * fix westend merge * impl on_swap for crowdloan * Check fund exists before create * update for crowdloan sig * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * slots on_initialize * use integration tests environment for benchmarks * fix hrmp event * auction on_initialize * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * fix storage name in auctions * add auction_index to winning data * winning data takes into account current auction index * remove println * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * Revert "add auction_index to winning data" * PastRandomness. * Fixes * Use new randomness * fix use of randomness in auctions and runtime config * expose consts * fix auction test * add deposit per byte for para registration * basic swap integration test * make swap test more comprehensive * Add WinningVec for easier retrieval in the front-end. * clean up `WinningVec` at the end * Add event for when a new best bid comes in * Fix propagation of winners in ending period * fix benchmarks, refund weight in dissolve * fix unused * remove some TODOs * setup opaque keys for paras in westend * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * remove unused * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * back to regular runtime config * use saturating math where user input can be * better first slot check * Update runtime/common/src/claims.rs * update westend onswap impl Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * update ring to 0.16.20 (#2626) * Remove TODO from substrate#2986 (#2628) * Companion for sub/8176 (#2622) * Merge * Fixes * Fix build * remove dep. * undo dep. * upadte substrate * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * Fix lock * revert lock; cargo update -p sp-io * from_rational_approx -> from_rational * Silence more warnings Co-authored-by: Gav Wood <gavin@parity.io> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * Add /data symlink to Docker containers (#2627) * add /data symlink to Docker * fix comments * Fix wrong deposit amount in council voters. (#2562) * Fix wrong deposit amount in council voters. * Fix some build * make it all compile.. so far. * Fix * break build * Okay fix it again * re benchmark (#2630) * Change something * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/ Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * Reduce number of active leaves at startup (#2631) Currently we will take all leaves and give that to the overseer on startup, but this is a bad idea when the finality is lagging for example. There can be many of unfinalized leaves, we don't even need to look at anymore. To solve this, the pr adds a maximum of 4 leaves we forward to the overseer and the pr also checks that we only pass uncles of the best block. * yet another set of logging improvements (#2638) * Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358) * service: update for substrate PR #7640 * update substrate * Add Pallet Babe to Integration Tests Runtime Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Companion for Substrate#8386 (#2634) * Companion for Substrate#8386 paritytech/substrate#8386 * "Update Substrate" Co-authored-by: parity-processbot <> * Request based collation fetching (#2621) * Introduce collation fetching protocol also move to mod.rs * Allow `PeerId`s in requests to network bridge. * Fix availability distribution tests. * Move CompressedPoV to primitives. * Request based collator protocol: validator side - Missing: tests - Collator side - don't connect, if not connected * Fixes. * Basic request based collator side. * Minor fix on collator side. * Don't connect in requests in collation protocol. Also some cleanup. * Fix PoV distribution * Bump substrate * Add back metrics + whitespace fixes. * Add back missing spans. * More cleanup. * Guide update. * Fix tests * Handle results in tests. * Fix weird compilation issue. * Add missing ) * Get rid of dead code. * Get rid of redundant import. * Fix runtime build. * Cleanup. * Fix wasm build. * Format fixes. Thanks @andronik ! * Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629) * Replace 'Module' with 'Pallet'. * "Update Substrate" * fix babe usage * fix benchmark Co-authored-by: parity-processbot <> Co-authored-by: thiolliere <gui.thiolliere@gmail.com> * integrate faster erasure code (#2608) Breaks compatibility for distributing PoV and PersistentValidationData between validators. Ref #2442 * Update for the new substrate client API (#2570) * Update for the new substrate client API * Code review suggestions * Update substrate * Improve the logging (#2645) * Don't accept incoming connections for collators (#2644) * Don't accept incoming connections for collators on the `Collation` peer set. * Better docs. * fix reconstruct fuzzer name * make script more robust * REVERTME: test run * REVERTME: test run II * Revert "REVERTME: test run II" This reverts commit 58375df. * Revert "REVERTME: test run" This reverts commit 9759cb6. Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Martin Pugh <pugh@s3kr.it> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com> Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Shaun Wang <spxwang@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

Related Substrate PR: paritytech/substrate#8386

* Bump Substrate * Change usage of "Module" to "Pallet" Related Substrate PR: paritytech/substrate#8372 * Add `OnSetCode` config param Related Substrate PR: paritytech/substrate#8496 * Update Aura Slot duration time type Related Substrate PR: paritytech/substrate#8386 * Add `OnSetCode` to mock runtimes * Add support for multiple justifications Related Substrate PR: paritytech/substrate#7640 * Use updated justification type in more places * Make GenesisConfig type non-optional Related Substrate PR: paritytech/substrate#8275 * Update service to use updated telemetry Related Substrate PR: paritytech/substrate#8143 * Appease Clippy

* Make slot duration being exposed as `Duration` to the outside * Some slot info love * Add `build_aura_worker` utility function * Copy copy copy

* Update branch * Make it compile * Compile * gate approval-checking logic (#2470) * Fix build * Updates * Fix merge * Adds missing crate * Companion for Substrate#8386 paritytech/substrate#8386 * Fix fix fix * Fix * Fix compilation * Rewrite to `ParachainsInherentDataProvider` * Make it compile * Renamings * Revert stuff * Remove stale file * Guide updates * Update node/core/parachains-inherent/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Update node/core/parachains-inherent/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Apply suggestions from code review * Reset accidental changes * More * Remove stale file * update Substrate Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: parity-processbot <>

* Make slot duration being exposed as `Duration` to the outside * Some slot info love * Add `build_aura_worker` utility function * Copy copy copy

* Update branch * Make it compile * Compile * gate approval-checking logic (#2470) * Fix build * Updates * Fix merge * Adds missing crate * Companion for Substrate#8386 paritytech/substrate#8386 * Fix fix fix * Fix * Fix compilation * Rewrite to `ParachainsInherentDataProvider` * Make it compile * Renamings * Revert stuff * Remove stale file * Guide updates * Update node/core/parachains-inherent/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Update node/core/parachains-inherent/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Apply suggestions from code review * Reset accidental changes * More * Remove stale file * update Substrate Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: parity-processbot <>

bkchr added 3 commits March 17, 2021 11:42

Make slot duration being exposed as Duration to the outside

58e22da

Some slot info love

313dcdf

Add build_aura_worker utility function

cb16666

bkchr added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Mar 17, 2021

bkchr requested review from andresilva and octol March 17, 2021 13:52

paritytech-ci added the D2-breaksapi label Mar 17, 2021

github-actions bot added the A7-needspolkadotpr label Mar 17, 2021

bkchr added a commit to paritytech/polkadot that referenced this pull request Mar 17, 2021

Companion for Substrate#8386

32501cc

paritytech/substrate#8386

bkchr mentioned this pull request Mar 17, 2021

Companion for Substrate#8386 paritytech/polkadot#2634

Merged

bkchr added the D1-audited 👍 PR contains changes to fund-managing logic that has been properly reviewed and externally audited label Mar 17, 2021

andresilva approved these changes Mar 17, 2021

View reviewed changes

bkchr added a commit to paritytech/polkadot that referenced this pull request Mar 17, 2021

Companion for Substrate#8386

780a34d

paritytech/substrate#8386

Copy copy copy

ce128c5

octol approved these changes Mar 17, 2021

View reviewed changes

ghost merged commit a0161f3 into master Mar 17, 2021

ghost deleted the bkchr-refactorings branch March 17, 2021 23:26

ghost pushed a commit to paritytech/polkadot that referenced this pull request Mar 17, 2021

Companion for Substrate#8386 (#2634)

402e1c0

* Companion for Substrate#8386 paritytech/substrate#8386 * "Update Substrate" Co-authored-by: parity-processbot <>

ordian pushed a commit to paritytech/polkadot that referenced this pull request Mar 19, 2021

Companion for Substrate#8386 (#2634)

6c2f47b

* Companion for Substrate#8386 paritytech/substrate#8386 * "Update Substrate" Co-authored-by: parity-processbot <>

HCastano added a commit to paritytech/parity-bridges-common that referenced this pull request Apr 6, 2021

Update Aura Slot duration time type

0ce48c5

Related Substrate PR: paritytech/substrate#8386

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aura and Slots refactoring #8386

Aura and Slots refactoring #8386

bkchr commented Mar 17, 2021 •

edited

bkchr commented Mar 17, 2021

andresilva Mar 17, 2021

bkchr Mar 17, 2021

andresilva Mar 17, 2021

octol Mar 17, 2021

andresilva commented Mar 17, 2021

ghost commented Mar 17, 2021

ghost commented Mar 17, 2021

bkchr commented Mar 17, 2021

ghost commented Mar 17, 2021

Aura and Slots refactoring #8386

Aura and Slots refactoring #8386

Conversation

bkchr commented Mar 17, 2021 • edited

bkchr commented Mar 17, 2021

andresilva Mar 17, 2021

Choose a reason for hiding this comment

bkchr Mar 17, 2021

Choose a reason for hiding this comment

andresilva Mar 17, 2021

Choose a reason for hiding this comment

octol Mar 17, 2021

Choose a reason for hiding this comment

andresilva commented Mar 17, 2021

ghost commented Mar 17, 2021

ghost commented Mar 17, 2021

bkchr commented Mar 17, 2021

ghost commented Mar 17, 2021

bkchr commented Mar 17, 2021 •

edited