-
Notifications
You must be signed in to change notification settings - Fork 242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
speed up ui tests. #944
speed up ui tests. #944
Conversation
7a91f00
to
a466d4d
Compare
* Update frame-metadata to v15.1.0 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Enable V15 unstable metadata in frame-metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Move validation hashing to dedicated file Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Use sp-metadata-ir from substrate to work with metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert using sp-metadata-ir in favor of conversion to v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Convert v14 to v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Use v15 for validation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Use v15 for codegen Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata/bench: Use v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust to v15 metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Improve documentation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * force CI Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * rpc: Fetch metadata at version Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * artifacts: Update polkadot.scale from commit 6dc9e84dde2 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Fetch V15 using the new API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Add runtime API interface Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Hash runtime API metadata for validation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Extract runtime API metadata wrapper from subxt::Metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Adjust hashing cache to reflect root+item keys Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * rpc: Add raw state_call API method Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Add payload with static and dynamic variants Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Allow payloads to call into the runtime Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * examples: Add example to make a runtime API call both static and dynamic Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Simplify client fetching Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Address feedback and fallback to old API if needed Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Make mutability conditional on input params Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Regenerate polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Retain only pallets without runtime API info Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Retry via `Metadata_metadata` without conversion Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * payload: Remove `Decode` and change validation fn Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Retain runtime API types Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Runtime APIs documentation based on flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update examples/examples/custom_metadata_url.rs Co-authored-by: James Wilson <james@jsdw.me> * Update artifacts from polkadot-a6cfdb16e9 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs with polkadot-a6cfdb16e9 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Generate input structures for runtime API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Remove the static paylaod and use single impl Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * examples: Fetch account nonce Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Adjust build script to fetch latest metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Check account nonce from runtime API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Fix doc generation for runtime types Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Rename `inputs` runtime calls module to `types` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Generate Calls structs inside the types module Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Check Alice account nonce before submitting the tx Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Add metadata version option flag supporting v14 and unstable Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Specify version to fetch Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Fallback to fetching latest stable metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Add unstable-metadata feature to fetch the latest Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * RuntimeVersion with Latest and Version(u32) Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Adjust fetch_metadata to inspect version list Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Adjust metadata to metadata_legacy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * events: Adjust docs to use metadata_legacy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * have a pass over fetch_metadata * cargo fmt * Option<String> when fetch metadata via latest API * clippy * fmt * cli: Use the MetadataVersion from codegen Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Specify latest as default for MetadataVersion Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Remove version from metadata and use the one from file_or_url Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Fix clippy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Decode metadata independently for different RPC calls Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: James Wilson <james@jsdw.me>
#947) * remove validation hash where not wanted * .into() * fix example * cargo fmt
* WIP Starting to write book; extrinsics first pass done * cargo fmt * Ongoing work; events, constants, wip blocks * at_latest() and wip blocks * remove need to import parity-scale-codec crate with Subxt for macro to work * More docs; expanding on setup guide and finish pass of main sections * Tidy and remove example section for now * format book lines to 100chars * Fix example code * cargo fmt * cargo fmt * fix example * Fix typos * fix broken doc links, pub mods * Update Subxt macro docs * can't link to Subxt here * move macro docs to Subxt to make linking better and fix example code * note on macro about docs * cargo fmt * document the no_default_derives macro feature * Address feedback and remove redundant text * address review comments; minor tweaks * WIP add Runtime calls to book * Improve Runtime API docs * expose thing we forgot to expose and doc link fixes
@tadeohepperle AFAIU the numbers you got on this branch was done with You didn't run the tests on this branch |
No, for this branch it is |
The reason I ask is because it would be good to know where perf improvements comes from i.e, whether it's Because |
I'm generally inclined to approve this because:
A couple of small outstanding questions to add to what Niklas said:
|
8d59910
to
82a5a21
Compare
subxt/src/book/usage/storage.rs
Outdated
@@ -18,7 +18,7 @@ | |||
//! ```rust,no_run | |||
//! use sp_keyring::AccountKeyring; | |||
//! | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata.scale")] | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could use the small metadata here?
subxt/src/book/setup/codegen.rs
Outdated
@@ -14,7 +14,7 @@ | |||
//! Using this macro looks something like: | |||
//! | |||
//! ```rust,no_run | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata.scale")] | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could use the tiny metadata here?
@@ -18,7 +18,7 @@ | |||
//! ```rust,no_run | |||
//! use sp_keyring::AccountKeyring; | |||
//! | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata.scale")] | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could use the small metadata here?
subxt/src/book/usage/runtime_apis.rs
Outdated
@@ -24,7 +24,7 @@ | |||
//! ```rust,no_run | |||
//! use sp_keyring::AccountKeyring; | |||
//! | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata.scale")] | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could use the tiny metadata here (I think)?
subxt/src/book/usage/storage.rs
Outdated
@@ -45,7 +45,7 @@ | |||
//! ```rust,no_run | |||
//! use sp_keyring::AccountKeyring; | |||
//! | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata.scale")] | |||
//! #[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale")] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: could use the small metadata here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! I think we could use a small/tiny metadata in a load of places where full
is used at the mo but we can tackle that in a follow up PR if you prefer (I commented on some but looks like many others could also be small or tiny)
@jsdw |
# to merge to master, too: | ||
branches: | ||
- master | ||
push: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hrm, you replaced tabs with spaces?
it makes this PR it a little noisy, would be better to make that change in another PR if needed.
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
# | ||
# This script is to be run from the root of the repository: `scripts/artifacts.sh` | ||
# | ||
# It expects a local polkadot node to be running a JSON-RPC HTTP server at 127.0.0.1:9933 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The server running on port 9933 is removed
# It expects a local polkadot node to be running a JSON-RPC HTTP server at 127.0.0.1:9933 | |
# It expects a local polkadot node to be running a JSON-RPC server at 127.0.0.1:9944 |
Partly fixes #877
Three changes were added here to speed up the tests:
Here are some runtime results from 4 runs on the master and the PR branch.
"All Other Tests" encompasses all tests except the wasm tests.
"UI Tests" is only the UI tests alone for the master branch. It is the job that takes most time and is a lower limit for the runtime of all non-wasm tests, if they run in parallel.
We can see that the speedup is at least about 30% (potentially much more, because all tests are compared to only the UI test job here)
The rows Test Wasm and Check Docs are added as control variables to see that nothing really changed there. Nextest does not do doc tests, so that stays the same.
With Speed Up Changes
Master