feat: upgrade to Polkadot v0.9.32 #434

wischli · 2022-11-14T15:42:25Z

fixes KILTProtocol/ticket#2289 and KILTProtocol/ticket#2296

Upgrades from Polkadot v0.9.29 to v0.9.32
Adds missing feature implementations for all tomls (checked via subalfred check features in all crates)
- Actually necessary for application of Improve usability of add+list_benchmark! paritytech/substrate#10592 (see runtime changes in dd81eac)
Migrates Democracy, Preimage and Scheduler pallets to use bounded Calls, see below

Summary of changes (Polkadot v0.9.30-0.9.32)

Weights v2 are not fully there yet, but the struct now includes the second field for storage size (tracking issue: The rest of the way to Weights V2 (Tracking Issue) paritytech/polkadot-sdk#256)

Breaking Changes

Breaking: Outer enums (BREAKING: Rename Call & Event paritytech/substrate#11981)
- Origin --> RuntimeOrigin
- Call --> RuntimeCall
- Event --> RuntimeEvent
- ~~Convention seems to be to keep Event, Call, Origin for inner pallet usage, e.g. Did::Origin~~ Update: We use Runtime prefix internally as well

Noteworthy PRs

Vesting pallet - make WithdrawReasons configurable paritytech/substrate#12109
rpc: Implement transaction RPC API paritytech/substrate#12328
Enable collation via RPC relay chain node paritytech/cumulus#1585
- Following the effort of decoupling collators and full relay nodes, this PR adds the possibility of pointing the collator to an “external” (non in-process) relay node. This is still considered experimental, and the relay node is suggested to run on the same machine than the collator for the moment.
- To specify the relay full node rpc: polkadot-parachain --alice --collator --relay-chain-rpc-url <rpc-websocket-url>
Enable the wasmtime-based WASM executor by default paritytech/substrate#12486
- Before this change only the interpreted WASM executor was included in per default compilations. Making the compiled executor opt-in, now, compiled WASM executor is set by default and an opt-out instead. This could lead to big performance difference between using these two, as more recent versions of the interpreter see a regression in performance.

Scheduler, Preimage, Democracy Migration

Bound uses of Call paritytech/substrate#11649
- Referenda, Democracy, Scheduler and Preimage pallets are all now bounded in storage access footprint
- Removed the concept of a "hard deadline" or weight-override priority and no longer guarantees that at least one scheduled item will be executed per block (since these are both dangerous to parachains which have a strict need of weight limits). This means you must ensure that scheduled items are below the MaximumWeight or they will not be executed.
Interesting comment: Bound uses of Call paritytech/substrate#11649 (comment)

There is migration code which moves existing proposals and referenda over to the new format. However IT DOES NOT MIGRATE EVERYTHING:

Preimages are NOT migrated. Any registered preimages in Democracy at the time of migration are dropped. Their balance is NOT UNRESERVED.

The re-dispatcher used in the old Democracy implementation is removed. Any proposals scheduled for dispatch by Democracy WILL NOT EXECUTE.

This means you SHOULD ensure that:

the preimage for the runtime upgrade is placed as an imminent preimage, not with a deposit;

no other preimages are in place at the time of upgrade;

there are no other proposals scheduled for dispatch by Democracy at the time of upgrade.

The Democracy pallet will be marked as deprecated immediately once Referenda is considered production-ready. ALL TEAMS ARE RECOMMENDED TO SWITCH SWAY FROM DEMOCRACY PALLET TO REFERENDA/CONVICTION-VOTING PALLETS ASAP

Result of `try-runtime` against Spiritnet on Friday Nov 18, 2022:

2022-11-18 09:27:23.917  INFO                 main runtime::preimage::migration::v1: Migrating 0 images
2022-11-18 09:27:23.917  INFO                 main runtime::scheduler::migration: Trying to migrate 0 agendas...
2022-11-18 09:27:23.917  INFO                 main runtime::scheduler::migration: Migrated 0 agendas.
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: 0 public proposals will be migrated.
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: 25 referenda will be migrated.
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#7
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#20
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#13
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#5
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#8
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#1
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#19
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#9
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#16
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#14
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#21
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#15
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#24
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#22
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#2
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#10
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum #0
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#6
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#11
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#3
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#17
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#18
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#23
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#4
2022-11-18 09:27:23.917  INFO                 main runtime::democracy::migration::v1: migrating referendum paritytech/substrate#25
2022-11-18 09:27:23.918  INFO                 main runtime::democracy::migration::v1: 0 public proposals migrated, 25 referenda migrated

Checklist:

I have verified that the code works
- No panics! (checked arithmetic ops, no indexing array[3] use get(3), ...)
I have verified that the code is easy to understand
- If not, I have left a well-balanced amount of inline comments
I have left the code in a better state
I have documented the changes (where applicable)

…tech/substrate/pull/11649

paritytech/substrate#11649

ntn-x2 · 2022-11-30T13:10:37Z

runtimes/standalone/src/lib.rs

-	type Event = Event;
-	type Call = Call;
+	type RuntimeEvent = RuntimeEvent;
+	type RuntimeCall = RuntimeCall;


The PR description says that the convetion seems to be to keep using Call, Event, Call etc for pallets, but I see here that that is not really the case. I guess we should also introduce the change in our pallets.

You are right. Will fix that!

ntn-x2 · 2022-11-30T13:11:40Z

runtimes/standalone/src/lib.rs

-	type Call = Call;
-	type Origin = Origin;
+	type RuntimeEvent = RuntimeEvent;
+	type Call = RuntimeCall;


The sudo pallet uses RuntimeCall, so I guess we should use the same here.

ntn-x2 · 2022-11-30T13:12:08Z

runtimes/standalone/src/lib.rs

-	type Origin = Origin;
+	type RuntimeEvent = RuntimeEvent;
+	type Call = RuntimeCall;
+	type Origin = RuntimeOrigin;


I am not sure at this point if keeping Origin makes sense anymore, or if we should also rename this to RuntimeOrigin.

ntn-x2 · 2022-11-30T13:13:40Z

runtimes/standalone/src/lib.rs

 /// Executive: handles dispatch to the various Pallets.
 pub type Executive =
 	frame_executive::Executive<Runtime, Block, frame_system::ChainContext<Runtime>, Runtime, AllPalletsWithSystem, ()>;

+#[cfg(feature = "runtime-benchmarks")]
+#[macro_use]
+extern crate frame_benchmarking;


Interesting.. Why do we need this here?

Can't this simply be done inside the benche module, since everything only exist for runtime-benchmarks?

We're just following upstream here. See this comment.

Can we add a comment in code why we have this there?

ntn-x2 · 2022-11-30T13:23:59Z

nodes/parachain/src/command.rs

@@ -336,6 +336,16 @@ pub fn run() -> Result<()> {
 					)?;
 					cmd.run(partials.client)
 				}),
+				#[cfg(not(feature = "runtime-benchmarks"))]
+				(BenchmarkCmd::Storage(_), _) => {
+					return Err(sc_cli::Error::Input(


Clippy noticed the return statement is unneeded.

ntn-x2 · 2022-11-30T13:27:05Z

pallets/pallet-did-lookup/src/migrations.rs

@@ -28,7 +28,9 @@ use frame_support::{
 use sp_std::{marker::PhantomData, vec};

 #[cfg(feature = "try-runtime")]
-use frame_support::traits::OnRuntimeUpgradeHelpersExt;
+use frame_support::codec::{Decode, Encode};


I guess these can be combined?

weichweich

Would like to have some insight on a few changes. Looks good otherwise. Nice set of changes that were included in the last updates.

pallets/pallet-did-lookup/src/migrations.rs

runtimes/common/src/constants.rs

runtimes/common/src/fees.rs

runtimes/clone/src/lib.rs

…0.9.32

wischli added 8 commits November 14, 2022 14:03

chore: bump rust to nightly-2022-10-09

f3c16ec

chore: bump deps

c2b7ece

wip: fix client

9f21da3

fix: client

5917b54

fix: max block weight, fee in runtime common

70475c7

refactor: runtime{call, event, origin}

20c2787

chore: init preimage scheduler migration\n\nhttps://github.com/parity…

5fb4855

…tech/substrate/pull/11649

wip: fix features

dd81eac

wischli self-assigned this Nov 16, 2022

wischli added 5 commits November 17, 2022 09:06

feat: add missing feats to missing tomls

ed2bb84

fix: clippy

f749e4a

tests: missing RuntimeCalls in runtime

19a24fc

fix: fees

a0e8f2d

chore: apply bound uses of call

909bb38

paritytech/substrate#11649

wischli requested review from ntn-x2 and weichweich November 18, 2022 15:37

wischli marked this pull request as ready for review November 18, 2022 15:37

Merge branch 'develop' into wf-2289-polkadot-v0.9.32

3e5cee3

ntn-x2 reviewed Nov 30, 2022

View reviewed changes

wischli mentioned this pull request Dec 5, 2022

feat: add block number for CTypes #440

Merged

weichweich approved these changes Dec 5, 2022

View reviewed changes

pallets/pallet-did-lookup/src/migrations.rs Outdated Show resolved Hide resolved

runtimes/common/src/constants.rs Show resolved Hide resolved

runtimes/common/src/fees.rs Outdated Show resolved Hide resolved

runtimes/clone/src/lib.rs Show resolved Hide resolved

wischli added 5 commits December 5, 2022 16:00

Merge remote-tracking branch 'origin/develop' into wf-2289-polkadot-v…

5f86576

…0.9.32

fix: apply suggestions from code review

91d932b

refactor: use runtime prefix for all events, origins, calls

fde113c

tests: use real block weights for fees

99eb1fd

docs: add comment for runtime bench block

f802641

wischli merged commit 431c1e9 into develop Dec 6, 2022

wischli deleted the wf-2289-polkadot-v0.9.32 branch December 6, 2022 15:39

ntn-x2 mentioned this pull request Jan 13, 2023

chore: upgrade polkadot to v0.9.36 #446

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: upgrade to Polkadot v0.9.32 #434

feat: upgrade to Polkadot v0.9.32 #434

wischli commented Nov 14, 2022 •

edited

Loading

ntn-x2 Nov 30, 2022

wischli Dec 5, 2022

wischli Dec 6, 2022

ntn-x2 Nov 30, 2022

wischli Dec 6, 2022

ntn-x2 Nov 30, 2022

wischli Dec 6, 2022

ntn-x2 Nov 30, 2022

ntn-x2 Nov 30, 2022

wischli Dec 5, 2022

weichweich Dec 6, 2022

wischli Dec 6, 2022

ntn-x2 Nov 30, 2022

wischli Dec 6, 2022

ntn-x2 Nov 30, 2022

wischli Dec 6, 2022 •

edited

Loading

weichweich left a comment

feat: upgrade to Polkadot v0.9.32 #434

feat: upgrade to Polkadot v0.9.32 #434

Conversation

wischli commented Nov 14, 2022 • edited Loading

fixes KILTProtocol/ticket#2289 and KILTProtocol/ticket#2296

Summary of changes (Polkadot v0.9.30-0.9.32)

Breaking Changes

Noteworthy PRs

Scheduler, Preimage, Democracy Migration

Result of try-runtime against Spiritnet on Friday Nov 18, 2022:

Checklist:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wischli Dec 6, 2022 • edited Loading

Choose a reason for hiding this comment

weichweich left a comment

Choose a reason for hiding this comment

wischli commented Nov 14, 2022 •

edited

Loading

Result of `try-runtime` against Spiritnet on Friday Nov 18, 2022:

wischli Dec 6, 2022 •

edited

Loading