Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ON HOLD] Rust Analyzer Benchmarking Infrastructure #1766

Draft
wants to merge 19 commits into
base: develop
Choose a base branch
from

Conversation

0xOmarA
Copy link
Member

@0xOmarA 0xOmarA commented Apr 8, 2024

Summary

INSTRUCTIONS:
Add summary - one or two sentences explaining the purpose of this PR.

Details

INSTRUCTIONS:
Provide further details about the changes, or how they fit into the roadmap.
You can delete this section if it's not useful.

Testing

INSTRUCTIONS:
Further details about the tests you've added or manually performed.
You can delete this section if it's not useful.

Update Recommendations

INSTRUCTIONS:
This section is to provide recommendations to consumers of this repo about how they
should handle breaking changes, or integrate new features. The two key stakeholder
groups are dApp Developers and Internal Integrators, and there are separate sections
for each.

In order to allow us to compile aggregated update instructions across PRs, please tag the PR
with 0+ of the relevant labels:
* scrypto-lib - Any change to the scrypto library
* sbor - Any breaking change to SBOR encoding/decoding
* manifest - Any change to manifest display, compilation/decompilation
* transaction - Any change which affects the compiled manifest, or transaction semantics
* substate - Any change to substates, the state model, or what's stored in the DB
* native-blueprint-interface - Any change to the interface of native blueprints
* files-moved - Any change to where engine types have moved, which will require
  internal integrators to update their imports

If you have a breaking change which doesn't fix into a category above, then tag it with
the closest label, and discuss in slack/discord.

If your PR contains no breaking changes or new features or hasn't been tagged, this whole
section can be deleted.

For dApp Developers

INSTRUCTIONS:
Migration recommendations for a dApp developer to update their dApp/integrations
due to to the change/s in this PR.

These will be aggregated by the Developer Ecosystem team and included in the next Scrypto migration guide
(eg https://docs-babylon.radixdlt.com/main/scrypto/release_notes/migrating_from_0.7_to_0.8.html )

For Internal Integrators

INSTRUCTIONS:
Instructions to any internal integrations (eg Node, Toolkit, Gateway, Ledger App) for how the changes may affect
them and recommendations for how they should update to support this change.

Copy link

github-actions bot commented Apr 8, 2024

Docker tags
docker.io/radixdlt/private-scrypto-builder:3c4e8425e1

Copy link

github-actions bot commented Apr 8, 2024

Benchmark for 3c4e842

Click to view benchmark
Test Base PR %
collection-autocomplete-no-scrypto-dependency 361.6±6.23ms N/A N/A
costing::bench_prepare_wasm 65.6±0.16ms 66.9±0.21ms +1.98%
costing::decode_sbor 11.0±0.07µs 11.1±0.06µs +0.91%
costing::decode_sbor_bytes 29.5±0.36µs 29.5±0.36µs 0.00%
costing::deserialize_wasm 1265.2±4.36µs 1278.2±5.03µs +1.03%
costing::instantiate_flash_loan 4.1±0.87ms 4.0±0.64ms -2.44%
costing::instantiate_radiswap 5.9±0.07ms 5.8±0.06ms -1.69%
costing::spin_loop 21.5±0.06ms 21.6±0.05ms +0.47%
costing::validate_sbor_payload 30.3±0.09µs 30.4±0.05µs +0.33%
costing::validate_sbor_payload_bytes 248.6±0.50ns 247.9±0.63ns -0.28%
costing::validate_secp256k1 76.2±0.07µs 76.3±0.05µs +0.13%
costing::validate_wasm 36.5±0.17ms 36.9±0.06ms +1.10%
decimal::add/0 8.5±0.02ns 8.4±0.00ns -1.18%
decimal::add/rust-native 9.9±0.01ns 9.8±0.00ns -1.01%
decimal::add/wasmer 112.3±0.15ns 111.2±0.05ns -0.98%
decimal::add/wasmer-call-native 453.1±0.27ns 474.4±1.45ns +4.70%
decimal::add/wasmi 623.9±2.02ns 624.0±2.73ns +0.02%
decimal::add/wasmi-call-native 5.0±0.00µs 5.5±0.01µs +10.00%
decimal::div/0 191.2±0.30ns 191.4±0.29ns +0.10%
decimal::from_string/0 158.0±0.44ns 152.0±0.39ns -3.80%
decimal::mul/0 142.9±0.09ns 141.1±0.19ns -1.26%
decimal::mul/rust-native 135.3±0.13ns 138.0±0.15ns +2.00%
decimal::mul/wasmer 1502.4±0.37ns 1525.7±3.06ns +1.55%
decimal::mul/wasmer-call-native 581.3±0.46ns 583.0±0.56ns +0.29%
decimal::mul/wasmi 41.6±0.16µs 41.8±0.15µs +0.48%
decimal::mul/wasmi-call-native 5.1±0.01µs 5.7±0.02µs +11.76%
decimal::pow/0 669.4±0.75ns 656.2±0.69ns -1.97%
decimal::pow/rust-native 629.3±0.84ns 632.0±0.25ns +0.43%
decimal::pow/wasmer 6.5±0.01µs 6.5±0.01µs 0.00%
decimal::pow/wasmer-call-native 1030.6±2.25ns 1031.8±2.94ns +0.12%
decimal::pow/wasmi 195.2±0.43µs 195.6±0.62µs +0.20%
decimal::pow/wasmi-call-native 5.0±0.01µs 5.5±0.02µs +10.00%
decimal::root/0 7.7±0.01µs 7.7±0.01µs 0.00%
decimal::sub/0 8.5±0.01ns 8.5±0.01ns 0.00%
decimal::to_string/0 452.9±0.85ns 440.8±0.52ns -2.67%
instantiation-method-inside-blueprint-no-scrypto-test-dependency-and-no-tests 1363.2±9.19ms N/A N/A
instantiation-method-inside-blueprint-with-scrypto-test-dependency-and-no-tests 4.2±0.03s N/A N/A
instantiation-method-inside-blueprint-with-scrypto-test-dependency-and-tests 4.3±0.03s N/A N/A
precise_decimal::add/0 9.4±0.03ns 9.5±0.07ns +1.06%
precise_decimal::add/rust-native 11.4±0.00ns 11.4±0.00ns 0.00%
precise_decimal::add/wasmer 119.3±0.11ns 114.8±0.93ns -3.77%
precise_decimal::add/wasmer-call-native 499.6±1.82ns 506.5±0.51ns +1.38%
precise_decimal::add/wasmi 788.0±1.23ns 801.8±11.01ns +1.75%
precise_decimal::add/wasmi-call-native 6.3±0.00µs 7.1±0.02µs +12.70%
precise_decimal::div/0 303.3±0.12ns 303.2±0.47ns -0.03%
precise_decimal::from_string/0 195.7±0.25ns 196.9±1.47ns +0.61%
precise_decimal::mul/0 344.4±0.78ns 345.5±1.14ns +0.32%
precise_decimal::mul/rust-native 304.3±0.29ns 302.8±1.86ns -0.49%
precise_decimal::mul/wasmer 3.4±0.00µs 3.5±0.00µs +2.94%
precise_decimal::mul/wasmer-call-native 826.7±0.76ns 824.8±1.07ns -0.23%
precise_decimal::mul/wasmi 105.8±0.62µs 105.8±0.14µs 0.00%
precise_decimal::mul/wasmi-call-native 6.8±0.00µs 7.5±0.01µs +10.29%
precise_decimal::pow/0 1876.7±6.17ns 1870.0±11.22ns -0.36%
precise_decimal::pow/rust-native 1483.8±2.12ns 1456.6±2.82ns -1.83%
precise_decimal::pow/wasmer 16.1±0.01µs 16.2±0.08µs +0.62%
precise_decimal::pow/wasmer-call-native 2.1±0.00µs 2.1±0.00µs 0.00%
precise_decimal::pow/wasmi 510.8±0.58µs 516.9±0.91µs +1.19%
precise_decimal::pow/wasmi-call-native 12.3±0.05µs 13.4±0.03µs +8.94%
precise_decimal::root/0 57.0±0.55µs 55.5±0.08µs -2.63%
precise_decimal::sub/0 9.5±0.00ns 9.5±0.01ns 0.00%
precise_decimal::to_string/0 722.4±1.89ns 752.2±2.99ns +4.13%
resource-builder-method-inside-blueprint-no-scrypto-test-dependency-and-no-tests 1368.4±10.25ms N/A N/A
resource-builder-method-inside-blueprint-with-scrypto-test-dependency-and-no-tests 4.2±0.04s N/A N/A
resource-builder-method-inside-blueprint-with-scrypto-test-dependency-and-tests 4.3±0.04s N/A N/A
resource-builder-method-outside-blueprint 1376.2±16.71ms N/A N/A
schema::validate_payload 343.8±0.57µs 387.5±0.30µs +12.71%
transaction::radiswap 5.3±0.02ms 5.3±0.02ms 0.00%
transaction::transfer 1758.5±10.78µs 1732.5±5.01µs -1.48%
transaction_processing::prepare 2.2±0.01ms 1468.0±2.17µs -33.27%
transaction_processing::prepare_and_decompile 6.0±0.22ms 5.5±0.48ms -8.33%
transaction_processing::prepare_and_decompile_and_recompile 24.1±0.83ms 25.0±1.70ms +3.73%
transaction_validation::validate_manifest 42.2±0.23µs 41.7±0.02µs -1.18%
transaction_validation::verify_bls_2KB 1058.9±105.40µs 1016.4±35.35µs -4.01%
transaction_validation::verify_bls_32B 1009.0±32.07µs 1047.3±70.64µs +3.80%
transaction_validation::verify_ecdsa 74.2±0.04µs 74.3±0.49µs +0.13%
transaction_validation::verify_ed25519 55.2±0.60µs 54.8±0.27µs -0.72%

@radixdlt radixdlt deleted a comment from github-actions bot Apr 13, 2024
Copy link

Autocomplete Benchmark Results

Test Name Autocomplete (ms)
benchmark_baseline 357
benchmark_component_instantiation_method_inside_blueprint_with_dev_dependencies 4174
benchmark_component_instantiation_method_inside_blueprint_without_dev_dependencies 1309
benchmark_resource_builder_method_inside_blueprint_with_dev_dependencies 4070
benchmark_resource_builder_method_inside_blueprint_without_dev_dependencies 1259
benchmark_resource_builder_method_outside_blueprint_with_dev_dependencies 4076
benchmark_resource_builder_method_outside_blueprint_without_dev_dependencies 1291

@0xOmarA 0xOmarA changed the title Feature/rust analyzer benchmarking Rust Analyzer Benchmarking Infrastructure Apr 13, 2024
@radixdlt radixdlt deleted a comment from github-actions bot Apr 13, 2024
Copy link

Autocomplete Benchmark Results

Test Name Autocomplete (ms)
benchmark_baseline 355
benchmark_component_instantiation_method_inside_blueprint_with_dev_dependencies 4147
benchmark_component_instantiation_method_inside_blueprint_without_dev_dependencies 1337
benchmark_resource_builder_method_inside_blueprint_with_dev_dependencies 4099
benchmark_resource_builder_method_inside_blueprint_without_dev_dependencies 1252
benchmark_resource_builder_method_outside_blueprint_with_dev_dependencies 4070
benchmark_resource_builder_method_outside_blueprint_without_dev_dependencies 1280

@dhedey dhedey changed the title Rust Analyzer Benchmarking Infrastructure [ON HOLD] Rust Analyzer Benchmarking Infrastructure Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant