XCM Benchmarks for Generic Instructions #3940

shawntabrizi · 2021-09-27T05:59:11Z

This PR introduces benchmarks for all "generic" XCM messages, that is, anything which does not use an AssetTransactor.

skip check-dependent-cumulus

shawntabrizi · 2021-10-04T06:10:26Z

/benchmark xcm pallet pallet_xcm_benchmarks::generic

parity-benchapp · 2021-10-04T06:10:27Z

Benchmark XCM for branch "shawntabrizi-xcm-generic-benchmarks" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/polkadot/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

Results

error[E0412]: cannot find type `Vec` in this scope
  --> xcm/pallet-xcm-benchmarks/src/generic/benchmarking.rs:81:16
   |
81 |         }).collect::<Vec<_>>();
   |                      ^^^ not found in this scope
   |
help: consider importing one of these items
   |
17 | use codec::alloc::vec::Vec;
   |
17 | use crate::Vec;
   |
17 | use frame_benchmarking::Vec;
   |
17 | use frame_support::inherent::Vec;
   |
     and 2 other candidates

error: aborting due to previous error

For more information about this error, try `rustc --explain E0412`.
error: could not compile `pallet-xcm-benchmarks`

To learn more, run the command again with --verbose.
error: build failed

ERROR: Unable to commit file ./runtime/polkadot/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

shawntabrizi · 2021-10-04T07:01:00Z

/benchmark xcm westend pallet_xcm_benchmarks::generic

parity-benchapp · 2021-10-04T07:01:02Z

Benchmark Westend XCM for branch "shawntabrizi-xcm-generic-benchmarks" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

Results

2021-10-04 07:13:18 [0] 💸 generated 1 npos voters, 1 from validators and 0 nominators    
2021-10-04 07:13:18 Took active validators from set with wrong size    
2021-10-04 07:13:18 Took active validators from set with wrong size.    
2021-10-04 07:13:18 Took active validators from set with wrong size    
2021-10-04 07:13:19 XCM ERROR >> Index: 0, Error: DestinationUnsupported, Weight: 0    
2021-10-04 07:13:19 XCM ERROR >> Index: 0, Error: DestinationUnsupported, Weight: 0    
Error: 
   0: �[91mInvalid input: xcm executor error: see error logs�[0m

Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

ERROR: Unable to commit file ./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

…k --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

…com/paritytech/polkadot into shawntabrizi-xcm-generic-benchmarks

shawntabrizi · 2021-12-01T06:28:57Z

@apopiak we should be good to go here

kianenigma · 2021-12-01T08:23:21Z

runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

@@ -1,73 +1,132 @@
-// this whole file is temp, and will be replaced in the future TODO
+// Copyright 2017-2021 Parity Technologies (UK) Ltd.


Suggested change

// Copyright 2017-2021 Parity Technologies (UK) Ltd.

// Copyright 2021 Parity Technologies (UK) Ltd.

kianenigma · 2021-12-01T08:34:54Z

xcm/pallet-xcm-benchmarks/src/generic/benchmarking.rs

+	}: {
+		executor.execute(xcm)?;
+	} verify {
+		// TODO make better assertion? #4426


you can write a watermark byte in the remark and make sure that it is inside the event as well.

kianenigma · 2021-12-01T08:37:02Z

xcm/pallet-xcm-benchmarks/src/generic/benchmarking.rs

+	} : {
+		executor.execute(xcm)?;
+	} verify {
+		assert_eq!(executor.origin, None);


what is this value before the benchmark?

xcm/pallet-xcm-benchmarks/src/generic/mock.rs

kianenigma

https://github.com/paritytech/polkadot/pull/3940/files#r759965208 might be a worthwhile consideration, otherwise LGTM

xcm/pallet-xcm-benchmarks/src/generic/mock.rs

apopiak

👍

Though I like Kian's suggestion of giving the parachain devs control over the benchmark error directly.

shawntabrizi · 2021-12-01T22:57:42Z

bot merge

paritytech-processbot · 2021-12-01T22:57:50Z

Waiting for commit status.

paritytech-processbot · 2021-12-01T23:03:55Z

Merge cancelled due to error. Error: Checks failed for 3c4dcb4

shawntabrizi · 2021-12-01T23:49:54Z

bot merge

paritytech-processbot · 2021-12-01T23:50:06Z

Waiting for commit status.

* master: (23 commits) Fix path of the polkadot_injected docker image (#4463) update docs on `validation_upgrade_frequency` (#4460) Fix path of the polkadot sha256 (#4458) Fix cumulus companion CI job (#4451) Add doc about runtime version bump (#4418) Bump trybuild from 1.0.52 to 1.0.53 (#4455) companion for #10231 (#4306) trivial fix (#4441) Rococo: ability to programatically assign slots to teams (#3943) Add parent header hash to log (#4421) bump tx versions (#4447) Companion for substrate/10347 (#4413) Improve paras runtime `BenchBuilder` api (#4318) Add CI team to `CODEOWNERS` file (#4350) XCM Benchmarks for Generic Instructions (#3940) Announce only on releases (#4417) Companion for 10379 (EnsureOneOf) (#4405) add pallet-babe/std (#4438) update Cargo.lock Squashed 'bridges/' changes from 407bf44a8a..1602249f0a ...

* cumulus: 4e95228 polkadot: 975e780ae0d988dc033f400ba822d14b326ee5b9 substrate: 89fcb3e4f62d221d4e161a437768e77d6265889e * fix refs * sync changes from paritytech/polkadot#3828 * sync changes from paritytech/polkadot#4387 * sync changes from paritytech/polkadot#3940 * sync with changes from paritytech/polkadot#4493 * sync with changes from paritytech/polkadot#4958 * sync with changes from paritytech/polkadot#3889 * sync with changes from paritytech/polkadot#5033 * sync with changes from paritytech/polkadot#5065 * compilation fixes * fixed prometheus endpoint startup (it now requires to be spawned within tokio context)

* Fix mandatory headers scanning in on-demand relay (#1306) * more logging in on-demand headerss * remove `maximal_headers_difference` concept from on-demand-relay * another leftover from previous on-demand version * removed extra log * fmt * increase relay balance guard limits for Polkadot<>Kusama bridge (#1308) * fix benchmarks before using it in Polkadot/Kusama/Rococo runtimes (#1309) * Endow relayer account at target chain in message benchmarks (#1310) * endow relayer account at target chain in message benchmarks * pick another line * expose fee multiplier metrics in messages relay (#1312) * Fix issues from cargo deny (#1311) * update libp2p-core (RUSTSEC-2022-0009) * update thread_local (RUSTSEC-2022-0006) * time 0.2 -> time 0.3 * ignore RUSTSEC-2021-0130 * proper migration to time 0.3 * fix clippy? * Revert "fix clippy?" This reverts commit 53bc289. * Add some tests to check integrity of chain constants + bridge configuration (#1316) * add some tests to check integrity of chain constants + bridge configuration * try to use named parameters where possible * Encode and estimate Rococo/Wococo/Kusama/Polkadot messages (#1322) * encode and estimate Rococo/Wococo/Kusama/Polkadot messages * allow send-message for non-bundled chains * weight -> dispatch-weight * fmt * fix spelling * impl Decode for SignedExtensions (otherwise transaction resubmitter panicks) (#1325) * use mortal transactions in transaction resubmitter (#1326) * Using-same-fork metric for finality and complex relay (#1327) * using_same_fork metric in finality relay * support `using_different_forks` in messages relay * added dashboards and alerts * lockfile * fix typo in alert expression (#1329) * removed extra *_RUNTIME_VERSION consts from relay code (#1330) * Reinitialize bridge relay subcommand (#1331) * reinitialize bridge subcommand * PolkadotToKusama in reinit-bridge * fix clippy issues (#1332) * Revert "Revert "override conversion rate in estimate-message-fee RPC (#1189)" (#1275)" (#1333) This reverts commit ed7def6. * Override conversion rate when computing message fee (#1261) * override conversion rate when message is sent * spelling + fmt * add --conversion-rate-override cli option * try to read conversion rate from cmd output * fix output * fmt * fi typo in generator script (#1334) * override conversion rate in tokens swap generator (#1335) * fix conversion rate override in token swap (#1336) * synchronize relay cli changes and token swap generator script (#1337) * fixed mess with conversion rates (#1338) * fix generator scripts to be consistent with updatedrelay output (#1339) * Increase rate from metric when estimating fee (#1340) * ignore errors when dumping logs and container is missing * fixed typo * print correct payload length * increase conversion rate a bit when estimating fee (to avoid message rejects when rate update tx is active) * fmt * fix conversion rate metric in dashboards (#1341) * get rid of '[No Data] Messages from Millau to Rialto are not being delivered' warnings (#1342) * use DecodeLimit when decoding incoming calls (#1344) * update regex to 1.5.5 (#1345) * edition = "2021" (#1346) * Mortal conversion rate updater transactions (#1257) * merge all similar update_conversion_rate functions * stall timeout in conversion rate update loop * fmt * fix * added alerts for relay balances (#1347) * replace From<>InboundLaneApi with direct storage reads (#1348) * added no_stack_overflow_when_decoding_nested_call_during_dispatch test (#1349) * ignore more "increase" alerts that are sometimes signalling NoData at startup (#1351) * Support dedicated lanes for pallets (#962) * pass call origin to the message verifier * is_outbound_lane_enabled -> is_message_accepted * trait SenderOrigin * only accept messages from token swap pallet to token swap lane * tests for edge cases of pay_delivery_and_dispatch_fee * fixed origin verification * fmt * fix benchmarks compilation * fix TODO with None account and non-zero message fee (already covered by tests) * revert cargo fmt changes temporarily * Update Substrate/Polkadot/Cumulus references (#1353) * cumulus: 4e95228 polkadot: 975e780ae0d988dc033f400ba822d14b326ee5b9 substrate: 89fcb3e4f62d221d4e161a437768e77d6265889e * fix refs * sync changes from paritytech/polkadot#3828 * sync changes from paritytech/polkadot#4387 * sync changes from paritytech/polkadot#3940 * sync with changes from paritytech/polkadot#4493 * sync with changes from paritytech/polkadot#4958 * sync with changes from paritytech/polkadot#3889 * sync with changes from paritytech/polkadot#5033 * sync with changes from paritytech/polkadot#5065 * compilation fixes * fixed prometheus endpoint startup (it now requires to be spawned within tokio context) * update chain versions (#1358) * update parity-scale-codec to 3.1.2 (#1359) * fix parse_transaction on Rialto+Millau (#1360)

initial stuff

16ad36c

shawntabrizi marked this pull request as draft September 27, 2021 05:59

shawntabrizi added the D9-needsaudit 👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited. label Sep 27, 2021

shawntabrizi added this to In progress in XCM via automation Sep 27, 2021

shawntabrizi added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Sep 27, 2021

shawntabrizi added 4 commits October 3, 2021 23:42

Merge branch 'master' into shawntabrizi-xcm-generic-benchmarks

cbd0dc8

quick fixes

82caeec

move to individual tests

e3a0f4a

dont need these

f5b3269

shawntabrizi added 2 commits October 4, 2021 02:16

Update benchmarking.rs

29bfa5d

add to westend

ec573a0

KiChjang mentioned this pull request Oct 27, 2021

XCM v3 #4097

Merged

27 tasks

shawntabrizi added 7 commits November 11, 2021 19:07

Merge branch 'master' into shawntabrizi-xcm-generic-benchmarks

e3a7c02

make benchmarks execute

77b1fbd

fix compile

470ce15

add post_execute

82149ac

ClaimAsset benchmark working

b817e7c

subscribe and unsubscribe benchmarks

3d9f774

benchmark for initiate reserve withdraw

e85c556

shawntabrizi marked this pull request as ready for review November 15, 2021 04:31

shawntabrizi requested review from acatangiu, adoerr, antonio-dropulic, svyatonik and tomusdrw as code owners November 15, 2021 04:31

Parity Bot and others added 3 commits December 1, 2021 06:23

fmt

df66c1f

Merge branch 'shawntabrizi-xcm-generic-benchmarks' of https://github.…

fc51779

…com/paritytech/polkadot into shawntabrizi-xcm-generic-benchmarks

one more todo needs to be labeled

cd0f4b8

kianenigma reviewed Dec 1, 2021

View reviewed changes

xcm/pallet-xcm-benchmarks/src/generic/mock.rs Outdated Show resolved Hide resolved

kianenigma approved these changes Dec 1, 2021

View reviewed changes

xcm/pallet-xcm-benchmarks/src/generic/mock.rs Outdated Show resolved Hide resolved

apopiak approved these changes Dec 1, 2021

View reviewed changes

shawntabrizi added 2 commits December 1, 2021 18:21

change option to result with benchmark error

b7e3ced

fmt and fix import

3c4dcb4

Merge branch 'master' into shawntabrizi-xcm-generic-benchmarks

a9ea6c1

paritytech-processbot bot merged commit 59aa955 into master Dec 1, 2021

XCM automation moved this from In progress to Done Dec 1, 2021

paritytech-processbot bot deleted the shawntabrizi-xcm-generic-benchmarks branch December 1, 2021 23:52

KiChjang mentioned this pull request Dec 20, 2021

Add Benchmarking Pipeline to XCM #3563

Closed

viniul added D1-audited 👍 PR contains changes to critical logic that has been properly reviewed and externally audited. and removed D9-needsaudit 👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited. labels Jan 6, 2022

shawntabrizi moved this from Done to Archive in XCM Jan 20, 2022

svyatonik added a commit to paritytech/parity-bridges-common that referenced this pull request Mar 15, 2022

sync changes from paritytech/polkadot#3940

a20c986

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XCM Benchmarks for Generic Instructions #3940

XCM Benchmarks for Generic Instructions #3940

shawntabrizi commented Sep 27, 2021 •

edited

Loading

shawntabrizi commented Oct 4, 2021

parity-benchapp bot commented Oct 4, 2021 •

edited

Loading

shawntabrizi commented Oct 4, 2021

parity-benchapp bot commented Oct 4, 2021 •

edited

Loading

shawntabrizi commented Dec 1, 2021

kianenigma Dec 1, 2021

kianenigma Dec 1, 2021

kianenigma Dec 1, 2021

kianenigma left a comment

apopiak left a comment

shawntabrizi commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

shawntabrizi commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

		@@ -1,73 +1,132 @@
		// this whole file is temp, and will be replaced in the future TODO
		// Copyright 2017-2021 Parity Technologies (UK) Ltd.

	// Copyright 2017-2021 Parity Technologies (UK) Ltd.
	// Copyright 2021 Parity Technologies (UK) Ltd.

XCM Benchmarks for Generic Instructions #3940

XCM Benchmarks for Generic Instructions #3940

Conversation

shawntabrizi commented Sep 27, 2021 • edited Loading

shawntabrizi commented Oct 4, 2021

parity-benchapp bot commented Oct 4, 2021 • edited Loading

shawntabrizi commented Oct 4, 2021

parity-benchapp bot commented Oct 4, 2021 • edited Loading

shawntabrizi commented Dec 1, 2021

kianenigma Dec 1, 2021

Choose a reason for hiding this comment

kianenigma Dec 1, 2021

Choose a reason for hiding this comment

kianenigma Dec 1, 2021

Choose a reason for hiding this comment

kianenigma left a comment

Choose a reason for hiding this comment

apopiak left a comment

Choose a reason for hiding this comment

shawntabrizi commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

shawntabrizi commented Dec 1, 2021

paritytech-processbot bot commented Dec 1, 2021

shawntabrizi commented Sep 27, 2021 •

edited

Loading

parity-benchapp bot commented Oct 4, 2021 •

edited

Loading

parity-benchapp bot commented Oct 4, 2021 •

edited

Loading