Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Engine executor to handle Wasm calls and returns #694

Merged
merged 49 commits into from
Feb 28, 2023

Conversation

Robbepop
Copy link
Member

No description provided.

@paritytech-cicd-pr
Copy link

paritytech-cicd-pr commented Feb 24, 2023

BENCHMARKS

NATIVEWASMTIME
BENCHMARKMASTERPRDIFFMASTERPRDIFFWASMTIME OVERHEAD
execute/
bare_call_0
1.44ms 1.87ms 🔴 30.44% 1.07ms 1.34ms 🔴 25.82% 🟢 -28%
execute/
bare_call_0/typed
1.02ms 1.12ms 🔴 9.51% 737.21µs 945.52µs 🔴 28.29% 🟢 -15%
execute/
bare_call_1
1.49ms 1.84ms 🔴 22.63% 1.18ms 1.40ms 🔴 18.09% 🟢 -24%
execute/
bare_call_16
2.32ms 2.49ms 🔴 6.17% 3.12ms 3.30ms 🔴 5.84% 🟢 33%
execute/
bare_call_16/typed
1.59ms 1.79ms 🔴 11.73% 1.52ms 1.55ms 🔴 2.14% 🟢 -13%
execute/
bare_call_1/typed
1.14ms 1.22ms 🔴 7.46% 986.90µs 1.22ms 🔴 23.25% 🟢 -1%
execute/
bare_call_4
1.56ms 1.75ms 🔴 12.20% 1.53ms 1.88ms 🔴 22.88% 🟢 7%
execute/
bare_call_4/typed
1.12ms 1.26ms 🔴 12.40% 1.03ms 1.14ms 🔴 10.94% 🟢 -9%
execute/
br_table
1.16ms 1.30ms 🔴 11.56% 1.21ms 1.38ms 🔴 14.68% 🟢 6%
execute/
count_until
650.11µs 680.20µs 🔴 4.65% 1.21ms 1.13ms 🟢 -7.01% 🟡 66%
execute/
factorial_iterative
321.56µs 335.81µs 🔴 4.46% 505.40µs 477.58µs 🟢 -5.47% 🟢 42%
execute/
factorial_recursive
622.92µs 595.73µs 🟢 -4.52% 1.09ms 953.44µs 🟢 -12.96% 🟡 60%
execute/
fib_iterative
1.46ms 1.47ms ⚪ 0.41% 2.45ms 2.23ms 🟢 -9.17% 🟡 52%
execute/
fib_recursive
5.74ms 4.80ms 🟢 -16.33% 9.70ms 7.76ms 🟢 -19.90% 🟡 62%
execute/
global_bump
892.73µs 823.45µs 🟢 -7.89% 2.26ms 1.56ms 🟢 -31.07% 🟡 90%
execute/
global_const
685.20µs 739.16µs 🔴 7.78% 1.45ms 1.43ms 🟢 -1.49% 🟡 94%
execute/
host_calls
27.48µs 31.72µs 🔴 15.31% 38.98µs 49.40µs 🔴 26.74% 🟡 56%
execute/
memory_fill
1.20ms 1.20ms ⚪ 0.32% 2.09ms 1.93ms 🟢 -7.40% 🟡 61%
execute/
memory_sum
1.26ms 1.18ms 🟢 -7.32% 2.14ms 1.93ms 🟢 -9.58% 🟡 63%
execute/
memory_vec_add
2.39ms 2.45ms 🔴 2.55% 4.91ms 4.59ms 🟢 -6.53% 🟡 87%
execute/
recursive_is_even
1.15ms 1.03ms 🟢 -10.10% 1.90ms 1.69ms 🟢 -10.73% 🟡 64%
execute/
recursive_ok
145.13µs 122.65µs 🟢 -15.38% 261.08µs 203.63µs 🟢 -22.02% 🟡 66%
execute/
recursive_scan
176.56µs 159.13µs 🟢 -9.88% 311.88µs 250.44µs 🟢 -19.74% 🟡 57%
execute/
recursive_trap
14.25µs 12.54µs 🟢 -11.94% 26.21µs 21.96µs 🟢 -16.12% 🟡 75%
execute/
regex_redux
530.13µs 520.04µs 🟢 -2.13% 1.01ms 911.31µs 🟢 -9.80% 🟡 75%
execute/
rev_complement
463.03µs 460.88µs ⚪ -0.46% 838.65µs 782.77µs 🟢 -6.64% 🟡 70%
execute/
tiny_keccak
328.92µs 324.64µs 🟢 -1.23% 641.50µs 603.33µs 🟢 -5.98% 🟡 86%
execute/
trunc_f2i
765.31µs 782.74µs 🔴 2.38% 1.44ms 1.40ms 🟢 -2.48% 🟡 79%
instantiate/
wasm_kernel
54.14µs 54.64µs ⚪ 2.04% 56.55µs 59.58µs 🔴 6.63% 🟢 9%
translate/
erc1155
234.92µs 233.67µs ⚪ -0.56% 441.19µs 438.96µs ⚪ -0.67% 🟡 88%
translate/
erc20
113.57µs 113.75µs ⚪ 0.30% 213.79µs 211.15µs ⚪ -1.08% 🟡 86%
translate/
erc721
160.91µs 161.78µs ⚪ 0.56% 311.76µs 307.21µs 🟢 -1.61% 🟡 90%
translate/
spidermonkey
0.00ns 0.00ns ⚪ 1.33% 0.00ns 0.00ns ⚪ -0.21% 🟢 0%
translate/
wasm_kernel
4.14ms 4.17ms ⚪ 1.19% 8.23ms 8.16ms ⚪ -1.12% 🟡 96%

Link to pipeline

@codecov-commenter
Copy link

codecov-commenter commented Feb 24, 2023

Codecov Report

Merging #694 (7906c9c) into master (43ce25d) will decrease coverage by 1.22%.
The diff coverage is 86.26%.

@@            Coverage Diff             @@
##           master     #694      +/-   ##
==========================================
- Coverage   81.01%   79.80%   -1.22%     
==========================================
  Files         100      100              
  Lines        8506     8203     -303     
==========================================
- Hits         6891     6546     -345     
- Misses       1615     1657      +42     
Impacted Files Coverage Δ
crates/wasmi/src/engine/stack/values/sp.rs 95.18% <ø> (ø)
crates/wasmi/src/engine/code_map.rs 82.50% <50.00%> (-1.72%) ⬇️
crates/wasmi/src/engine/cache.rs 81.25% <71.42%> (-0.57%) ⬇️
crates/wasmi/src/engine/stack/mod.rs 74.64% <82.35%> (-7.18%) ⬇️
crates/wasmi/src/engine/mod.rs 81.72% <84.84%> (-2.14%) ⬇️
crates/wasmi/src/engine/stack/values/tests.rs 97.72% <87.50%> (-2.28%) ⬇️
crates/wasmi/src/engine/stack/values/mod.rs 86.84% <92.30%> (-0.34%) ⬇️
crates/wasmi/src/engine/executor.rs 86.52% <93.45%> (-8.56%) ⬇️
crates/wasmi/src/engine/stack/frames.rs 100.00% <100.00%> (ø)
... and 2 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@Robbepop Robbepop marked this pull request as draft February 26, 2023 09:51
@Robbepop Robbepop marked this pull request as ready for review February 28, 2023 00:13
@Robbepop
Copy link
Member Author

With respect to benchmarks it is expected that host call performance regresses with this PR.
That said it is not (yet) clear to me why bare_calls, global_const, count_until and factorial_iterative regress slightly on native.
However, those minor regressions are outweight by the impressive performance gains on both native and Wasm especially for recursive work loads.
Therefore, I propose to finally merge this PR and research the aforementioned performance regressions in follow-up PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants