Add benchmarking for parachain runtime `ump` pallet #3889

hirschenberger · 2021-09-18T09:43:54Z

Part of #3850

@KiChjang Can you add me to the tracking list?

Polkadot address: 1ZApdbwrNnT1Hey4xeTAH6SZPDgbA9ZVftNuu9jTRbkc2Bg

hirschenberger · 2021-10-11T20:26:08Z

/benchmark runtime westend runtime_parachains::ump

parity-benchapp · 2021-10-11T20:26:10Z

Benchmark Runtime Westend Pallet for branch "issue_3850_ump" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::ump --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_ump.rs

Results

Pallet: "runtime_parachains::ump", Extrinsic: "service_overweight", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Ump Overweight (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    44.51
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    44.51
              µs

Reads = 1
Writes = 1

parity-benchapp · 2021-10-12T05:43:56Z

Error running benchmark: issue_3850_ump

stdout

Caught exception in benchmarkRuntime: TypeError: Cannot read property 'benchCommand' of undefined

hirschenberger · 2021-10-12T05:45:10Z

/benchmark runtime kusama runtime_parachains::ump

parity-benchapp · 2021-10-12T05:45:11Z

Benchmark Runtime Kusama Pallet for branch "issue_3850_ump" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::ump --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_ump.rs

Results

Pallet: "runtime_parachains::ump", Extrinsic: "service_overweight", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Ump Overweight (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    34.99
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    34.99
              µs

Reads = 1
Writes = 1

…issue_3850_ump

…k --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::ump --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_ump.rs

…nto issue_3850_ump

drahnr

Generally LGTM, I defer the final review to @pepyakin who knows the intricacies and possible side effects better, and hence can judge if the benchmarks measure what the ought to measure :)

pepyakin

Looks good. I am OK with merging it as is, possibly following up with nit fixes. Thanks for your work and I am sorry about it took so much time to get the review.

pepyakin · 2022-02-08T14:48:08Z

runtime/parachains/src/ump.rs

 	}

 	/// Remove all relevant storage items for an outgoing parachain.
-	fn clean_ump_after_outgoing(outgoing_para: &ParaId) {
+	pub(crate) fn clean_ump_after_outgoing(outgoing_para: &ParaId) -> Weight {


Nit: Why is this pub(crate) though? Can we constrain this to pub(in benchmarking) or in a similar fashion?

When I do pub(in crate::ump::benchmarking) the compiler complains: error[E0742]: visibilities can only be restricted to ancestor modules

runtime/parachains/src/ump.rs

runtime/parachains/src/ump/benchmarking.rs

runtime/parachains/src/ump.rs

pepyakin · 2022-02-08T14:58:28Z

runtime/parachains/src/ump/benchmarking.rs

+
+fn create_message_size<T: Config>(size: u32) -> Vec<u8> {
+	// fill the remark but leave some bytes for the encoding of the message
+	let size = size.saturating_sub(20) as usize;


What is this 20 exactly? Can you elaborate in the comment? Ideally we should be able to catch a case when this 20 becomes something else

I changed the function name to indicate that the created message will have at least size bytes.

I also changed the code to create an empty message (without remark payload) to determine the encoded size and use this to create the actual message as close as possible (but always bigger) to size. Ensuring that with an assertion.

drahnr · 2022-02-08T17:44:33Z

Let's merge this after 0.9.17 is cut.

hirschenberger · 2022-02-08T17:47:05Z

Ok, I'll fix the nits first.

emostov · 2022-02-25T18:58:11Z

Let's merge this after 0.9.17 is cut.

@drahnr is this ok to merge now?

drahnr · 2022-02-25T19:29:26Z

Yes, feel free to merge

kianenigma · 2022-02-26T08:52:46Z

bot merge

kianenigma · 2022-02-26T08:53:18Z

@hirschenberger can you put your DOT or KSM address in your PR description? Then we can propose a tip for you!

hirschenberger · 2022-02-26T10:41:50Z

@hirschenberger can you put your DOT or KSM address in your PR description? Then we can propose a tip for you!

Oh nice, thank you. I'm glad I got this PR finally landed. My DOT address is:
1ZApdbwrNnT1Hey4xeTAH6SZPDgbA9ZVftNuu9jTRbkc2Bg

shawntabrizi · 2022-02-26T17:40:21Z

/tip medium

substrate-tip-bot · 2022-02-26T17:40:25Z

A medium tip was successfully submitted for hirschenberger (1ZApdbwrNnT1Hey4xeTAH6SZPDgbA9ZVftNuu9jTRbkc2Bg on polkadot).

https://polkadot.js.org/apps/#/treasury/tips

* cumulus: 4e95228 polkadot: 975e780ae0d988dc033f400ba822d14b326ee5b9 substrate: 89fcb3e4f62d221d4e161a437768e77d6265889e * fix refs * sync changes from paritytech/polkadot#3828 * sync changes from paritytech/polkadot#4387 * sync changes from paritytech/polkadot#3940 * sync with changes from paritytech/polkadot#4493 * sync with changes from paritytech/polkadot#4958 * sync with changes from paritytech/polkadot#3889 * sync with changes from paritytech/polkadot#5033 * sync with changes from paritytech/polkadot#5065 * compilation fixes * fixed prometheus endpoint startup (it now requires to be spawned within tokio context)

* Fix mandatory headers scanning in on-demand relay (#1306) * more logging in on-demand headerss * remove `maximal_headers_difference` concept from on-demand-relay * another leftover from previous on-demand version * removed extra log * fmt * increase relay balance guard limits for Polkadot<>Kusama bridge (#1308) * fix benchmarks before using it in Polkadot/Kusama/Rococo runtimes (#1309) * Endow relayer account at target chain in message benchmarks (#1310) * endow relayer account at target chain in message benchmarks * pick another line * expose fee multiplier metrics in messages relay (#1312) * Fix issues from cargo deny (#1311) * update libp2p-core (RUSTSEC-2022-0009) * update thread_local (RUSTSEC-2022-0006) * time 0.2 -> time 0.3 * ignore RUSTSEC-2021-0130 * proper migration to time 0.3 * fix clippy? * Revert "fix clippy?" This reverts commit 53bc289. * Add some tests to check integrity of chain constants + bridge configuration (#1316) * add some tests to check integrity of chain constants + bridge configuration * try to use named parameters where possible * Encode and estimate Rococo/Wococo/Kusama/Polkadot messages (#1322) * encode and estimate Rococo/Wococo/Kusama/Polkadot messages * allow send-message for non-bundled chains * weight -> dispatch-weight * fmt * fix spelling * impl Decode for SignedExtensions (otherwise transaction resubmitter panicks) (#1325) * use mortal transactions in transaction resubmitter (#1326) * Using-same-fork metric for finality and complex relay (#1327) * using_same_fork metric in finality relay * support `using_different_forks` in messages relay * added dashboards and alerts * lockfile * fix typo in alert expression (#1329) * removed extra *_RUNTIME_VERSION consts from relay code (#1330) * Reinitialize bridge relay subcommand (#1331) * reinitialize bridge subcommand * PolkadotToKusama in reinit-bridge * fix clippy issues (#1332) * Revert "Revert "override conversion rate in estimate-message-fee RPC (#1189)" (#1275)" (#1333) This reverts commit ed7def6. * Override conversion rate when computing message fee (#1261) * override conversion rate when message is sent * spelling + fmt * add --conversion-rate-override cli option * try to read conversion rate from cmd output * fix output * fmt * fi typo in generator script (#1334) * override conversion rate in tokens swap generator (#1335) * fix conversion rate override in token swap (#1336) * synchronize relay cli changes and token swap generator script (#1337) * fixed mess with conversion rates (#1338) * fix generator scripts to be consistent with updatedrelay output (#1339) * Increase rate from metric when estimating fee (#1340) * ignore errors when dumping logs and container is missing * fixed typo * print correct payload length * increase conversion rate a bit when estimating fee (to avoid message rejects when rate update tx is active) * fmt * fix conversion rate metric in dashboards (#1341) * get rid of '[No Data] Messages from Millau to Rialto are not being delivered' warnings (#1342) * use DecodeLimit when decoding incoming calls (#1344) * update regex to 1.5.5 (#1345) * edition = "2021" (#1346) * Mortal conversion rate updater transactions (#1257) * merge all similar update_conversion_rate functions * stall timeout in conversion rate update loop * fmt * fix * added alerts for relay balances (#1347) * replace From<>InboundLaneApi with direct storage reads (#1348) * added no_stack_overflow_when_decoding_nested_call_during_dispatch test (#1349) * ignore more "increase" alerts that are sometimes signalling NoData at startup (#1351) * Support dedicated lanes for pallets (#962) * pass call origin to the message verifier * is_outbound_lane_enabled -> is_message_accepted * trait SenderOrigin * only accept messages from token swap pallet to token swap lane * tests for edge cases of pay_delivery_and_dispatch_fee * fixed origin verification * fmt * fix benchmarks compilation * fix TODO with None account and non-zero message fee (already covered by tests) * revert cargo fmt changes temporarily * Update Substrate/Polkadot/Cumulus references (#1353) * cumulus: 4e95228 polkadot: 975e780ae0d988dc033f400ba822d14b326ee5b9 substrate: 89fcb3e4f62d221d4e161a437768e77d6265889e * fix refs * sync changes from paritytech/polkadot#3828 * sync changes from paritytech/polkadot#4387 * sync changes from paritytech/polkadot#3940 * sync with changes from paritytech/polkadot#4493 * sync with changes from paritytech/polkadot#4958 * sync with changes from paritytech/polkadot#3889 * sync with changes from paritytech/polkadot#5033 * sync with changes from paritytech/polkadot#5065 * compilation fixes * fixed prometheus endpoint startup (it now requires to be spawned within tokio context) * update chain versions (#1358) * update parity-scale-codec to 3.1.2 (#1359) * fix parse_transaction on Rialto+Millau (#1360)

Claim grounds, not working yet.

b5b483d

KiChjang mentioned this pull request Sep 18, 2021

Benchmark Parachains Runtime: Tracking Issue #3850

Closed

14 tasks

hirschenberger added 3 commits September 27, 2021 14:32

Merge remote-tracking branch 'origin/master' into issue_3850_ump

80c429b

Merge remote-tracking branch 'origin/master' into issue_3850_ump

ab8669c

Merge remote-tracking branch 'origin/master' into issue_3850_ump

f76f68f

hirschenberger marked this pull request as draft September 28, 2021 14:25

hirschenberger added 8 commits September 29, 2021 10:17

still wip, nothing to see

ce8a9d7

Merge remote-tracking branch 'origin/master' into issue_3850_ump

ec01972

Merge remote-tracking branch 'origin/master' into issue_3850_ump

d0c192e

Module visibility problem

e5be79e

Merge remote-tracking branch 'origin/master' into issue_3850_ump

a67cf3f

Merge remote-tracking branch 'origin/master' into issue_3850_ump

021b161

Add benchmarks for all parachain-enabled chains

b33272c

Add missing mock weights

a3330da

hirschenberger force-pushed the issue_3850_ump branch 3 times, most recently from 9d51dd9 to 35ece1a Compare October 11, 2021 16:11

Fix mock weight info

06cfaba

hirschenberger force-pushed the issue_3850_ump branch from 35ece1a to 06cfaba Compare October 11, 2021 16:21

hirschenberger marked this pull request as ready for review October 11, 2021 20:19

Add max-weight to extrinsic weight

4fea1a2

hirschenberger force-pushed the issue_3850_ump branch from 350f5c5 to 4fea1a2 Compare October 12, 2021 05:04

Parity Bot and others added 3 commits October 12, 2021 05:45

Merge branch 'master' of https://github.com/paritytech/polkadot into …

664d159

…issue_3850_ump

sorting declarations

29f22a6

hirschenberger added 5 commits February 4, 2022 02:32

Merge remote-tracking branch 'origin/master' into issue_3850_ump

9be975e

Better variable naming

c75aa20

Merge branch 'issue_3850_ump' of github.com:hirschenberger/polkadot i…

7938978

…nto issue_3850_ump

Fix build

3e98695

Fix build

1c62492

hirschenberger force-pushed the issue_3850_ump branch from f10ee10 to 3e98695 Compare February 4, 2022 03:02

Merge branch 'issue_3850_ump' of github.com:hirschenberger/polkadot i…

799f9a1

…nto issue_3850_ump

hirschenberger requested a review from drahnr February 4, 2022 11:10

drahnr modified the milestones: v0.9.18 : Polkadot disputes, v0.9.19 Feb 4, 2022

drahnr reviewed Feb 4, 2022

View reviewed changes

pepyakin approved these changes Feb 8, 2022

View reviewed changes

Fix some remarks from @pepyakin

441b6c4

apopiak removed their request for review February 14, 2022 10:11

Merge remote-tracking branch 'upstream/master' into issue_3850_ump

780d40f

paritytech-processbot bot merged commit b567c8b into paritytech:master Feb 26, 2022

svyatonik added a commit to paritytech/parity-bridges-common that referenced this pull request Mar 15, 2022

sync with changes from paritytech/polkadot#3889

c44db85

coderobe mentioned this pull request Apr 8, 2022

Polkadot v0.9.19 Release checklist #5288

Closed

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add benchmarking for parachain runtime `ump` pallet #3889

Add benchmarking for parachain runtime `ump` pallet #3889

hirschenberger commented Sep 18, 2021 •

edited by KiChjang

Loading

hirschenberger commented Oct 11, 2021

parity-benchapp bot commented Oct 11, 2021 •

edited

Loading

parity-benchapp bot commented Oct 12, 2021 •

edited

Loading

hirschenberger commented Oct 12, 2021

parity-benchapp bot commented Oct 12, 2021 •

edited

Loading

drahnr left a comment •

edited

Loading

pepyakin left a comment

pepyakin Feb 8, 2022

hirschenberger Feb 8, 2022

pepyakin Feb 8, 2022

hirschenberger Feb 9, 2022

drahnr commented Feb 8, 2022

hirschenberger commented Feb 8, 2022

emostov commented Feb 25, 2022

drahnr commented Feb 25, 2022

kianenigma commented Feb 26, 2022

kianenigma commented Feb 26, 2022

hirschenberger commented Feb 26, 2022

shawntabrizi commented Feb 26, 2022

substrate-tip-bot bot commented Feb 26, 2022

Add benchmarking for parachain runtime ump pallet #3889

Add benchmarking for parachain runtime ump pallet #3889

Conversation

hirschenberger commented Sep 18, 2021 • edited by KiChjang Loading

hirschenberger commented Oct 11, 2021

parity-benchapp bot commented Oct 11, 2021 • edited Loading

parity-benchapp bot commented Oct 12, 2021 • edited Loading

hirschenberger commented Oct 12, 2021

parity-benchapp bot commented Oct 12, 2021 • edited Loading

drahnr left a comment • edited Loading

Choose a reason for hiding this comment

pepyakin left a comment

Choose a reason for hiding this comment

pepyakin Feb 8, 2022

Choose a reason for hiding this comment

hirschenberger Feb 8, 2022

Choose a reason for hiding this comment

pepyakin Feb 8, 2022

Choose a reason for hiding this comment

hirschenberger Feb 9, 2022

Choose a reason for hiding this comment

drahnr commented Feb 8, 2022

hirschenberger commented Feb 8, 2022

emostov commented Feb 25, 2022

drahnr commented Feb 25, 2022

kianenigma commented Feb 26, 2022

kianenigma commented Feb 26, 2022

hirschenberger commented Feb 26, 2022

shawntabrizi commented Feb 26, 2022

substrate-tip-bot bot commented Feb 26, 2022

Add benchmarking for parachain runtime `ump` pallet #3889

Add benchmarking for parachain runtime `ump` pallet #3889

hirschenberger commented Sep 18, 2021 •

edited by KiChjang

Loading

parity-benchapp bot commented Oct 11, 2021 •

edited

Loading

parity-benchapp bot commented Oct 12, 2021 •

edited

Loading

parity-benchapp bot commented Oct 12, 2021 •

edited

Loading

drahnr left a comment •

edited

Loading