refactor(sandbox-host): Process wasm-globals on wasmer side #2969

gshep · 2023-07-20T16:18:44Z

Resolves #2920

@gear-tech/dev

Release notes: This PR introduces processing of WASM-globals on execution backend side (namely wasmer). The approach allows to eliminate redundant host-calls (what is expensive in terms of gas consumption) to get/set the binary globals during execution of a program. The benchmarks show syscalls gas reducing on 30% in average, up to 90% at most.

gshep · 2023-07-23T14:02:20Z

from Benchmarks 84 - https://github.com/gear-tech/gear/actions/runs/5635224234

Comparison table for Gear runtime

| name                                    | master   | gshep/issue-2920-2 | diff     |
|-----------------------------------------|----------|--------------------|----------|
| alloc                                   | 8.1 µs   | 6.9 µs             | +18.37%  |
| alloc_per_page                          | 426.6 ns | 411.9 ns           | +3.55%   |
| free                                    | 2.1 µs   | 951.0 ns           | +121.69% |
| gr_reserve_gas                          | 4.3 µs   | 3.2 µs             | +34.76%  |
| gr_unreserve_gas                        | 4.7 µs   | 3.6 µs             | +32.41%  |
| gr_system_reserve_gas                   | 2.9 µs   | 1.8 µs             | +66.13%  |
| gr_gas_available                        | 2.9 µs   | 1.7 µs             | +67.88%  |
| gr_message_id                           | 2.9 µs   | 1.7 µs             | +71.06%  |
| gr_pay_program_rent                     | 50.6 ns  | 36.3 ns            | +39.44%  |
| gr_program_id                           | 2.9 µs   | 1.7 µs             | +70.77%  |
| gr_source                               | 2.9 µs   | 1.7 µs             | +69.82%  |
| gr_value                                | 2.9 µs   | 1.7 µs             | +73.19%  |
| gr_value_available                      | 2.9 µs   | 1.7 µs             | +69.73%  |
| gr_size                                 | 3.0 µs   | 1.7 µs             | +69.90%  |
| gr_read                                 | 4.0 µs   | 2.8 µs             | +39.63%  |
| gr_read_per_byte                        | 162 ps   | 161 ps             | +0.62%   |
| gr_block_height                         | 2.9 µs   | 1.7 µs             | +69.81%  |
| gr_block_timestamp                      | 2.9 µs   | 1.7 µs             | +67.30%  |
| gr_random                               | 3.7 µs   | 2.6 µs             | +42.60%  |
| gr_reply_deposit                        | 8.7 µs   | 7.7 µs             | +12.11%  |
| gr_send                                 | 5.6 µs   | 4.3 µs             | +29.67%  |
| gr_send_per_byte                        | 262 ps   | 303 ps             | -13.53%  |
| gr_send_wgas                            | 5.6 µs   | 4.4 µs             | +28.46%  |
| gr_send_wgas_per_byte                   | 262 ps   | 302 ps             | -13.25%  |
| gr_send_init                            | 3.1 µs   | 1.9 µs             | +63.44%  |
| gr_send_push                            | 4.2 µs   | 3.2 µs             | +31.84%  |
| gr_send_push_per_byte                   | 381 ps   | 421 ps             | -9.50%   |
| gr_send_commit                          | 4.9 µs   | 3.8 µs             | +27.84%  |
| gr_send_commit_wgas                     | 5.1 µs   | 3.9 µs             | +31.53%  |
| gr_reservation_send                     | 5.7 µs   | 4.5 µs             | +28.31%  |
| gr_reservation_send_per_byte            | 258 ps   | 304 ps             | -15.13%  |
| gr_reservation_send_commit              | 5.1 µs   | 4.1 µs             | +26.07%  |
| gr_reply_commit                         | 20.0 µs  | 21.5 µs            | -6.70%   |
| gr_reply_commit_wgas                    | 21.3 µs  | 15.2 µs            | +39.79%  |
| gr_reservation_reply                    | 12.4 µs  | 8.2 µs             | +51.90%  |
| gr_reservation_reply_per_byte           | 428.1 ns | 481.7 ns           | -11.12%  |
| gr_reservation_reply_commit             | 12.5 µs  | 9.5 µs             | +31.29%  |
| gr_reply_push                           | 4.1 µs   | 3.1 µs             | +33.37%  |
| gr_reply                                | 20.8 µs  | 18.4 µs            | +13.46%  |
| gr_reply_per_byte                       | 417 ps   | 465 ps             | -10.32%  |
| gr_reply_wgas                           | 19.1 µs  | 14.0 µs            | +36.11%  |
| gr_reply_wgas_per_byte                  | 414 ps   | 470 ps             | -11.91%  |
| gr_reply_push_per_byte                  | 628 ps   | 673 ps             | -6.69%   |
| gr_reply_to                             | 2.9 µs   | 1.8 µs             | +67.51%  |
| gr_signal_from                          | 2.9 µs   | 1.7 µs             | +71.70%  |
| gr_reply_input                          | 36.3 µs  | 10.9 µs            | +233.78% |
| gr_reply_input_wgas                     | 1.8 µs   | 9.6 µs             | -81.48%  |
| gr_reply_push_input                     | 3.1 µs   | 2.0 µs             | +51.90%  |
| gr_reply_push_input_per_byte            | 143 ps   | 106 ps             | +34.91%  |
| gr_send_input                           | 5.8 µs   | 4.5 µs             | +26.80%  |
| gr_send_input_wgas                      | 5.8 µs   | 4.6 µs             | +26.48%  |
| gr_send_push_input                      | 3.3 µs   | 2.2 µs             | +50.36%  |
| gr_send_push_input_per_byte             | 166 ps   | 150 ps             | +10.67%  |
| gr_debug                                | 3.1 µs   | 2.0 µs             | +56.79%  |
| gr_debug_per_byte                       | 311 ps   | 371 ps             | -16.17%  |
| gr_reply_code                           | 2.9 µs   | 1.7 µs             | +70.82%  |
| gr_exit                                 | 30.0 µs  | 20.6 µs            | +45.48%  |
| gr_leave                                | 17.1 µs  | 14.0 µs            | +22.25%  |
| gr_wait                                 | 15.2 µs  | 13.1 µs            | +16.22%  |
| gr_wait_for                             | 15.7 µs  | 8.6 µs             | +82.71%  |
| gr_wait_up_to                           | 14.5 µs  | 12.8 µs            | +13.22%  |
| gr_wake                                 | 4.2 µs   | 3.2 µs             | +31.86%  |
| gr_create_program                       | 6.6 µs   | 5.4 µs             | +22.15%  |
| gr_create_program_payload_per_byte      | 88 ps    | 115 ps             | -23.48%  |
| gr_create_program_salt_per_byte         | 2.1 ns   | 2.2 ns             | -2.10%   |
| gr_create_program_wgas                  | 6.7 µs   | 5.5 µs             | +21.12%  |
| gr_create_program_wgas_payload_per_byte | 84 ps    | 112 ps             | -25.00%  |
| gr_create_program_wgas_salt_per_byte    | 2.2 ns   | 2.2 ns             | -1.78%   |

gshep · 2023-07-24T07:17:27Z

blocked by #2948

ark0f · 2023-08-02T20:54:29Z

I guess it's a part of #1771?

gshep · 2023-08-07T10:14:33Z

I guess it's a part of #1771?

Nope, 1771 is a different approach involving modified instrumentation.

Conflicts: core-backend/common/src/funcs.rs core-backend/sandbox/src/env.rs core-backend/sandbox/src/runtime.rs core-backend/wasmi/src/runtime.rs

sandbox/host/src/sandbox/wasmer_backend.rs

core-backend/common/src/funcs.rs

core-backend/sandbox/src/runtime.rs

sandbox/host/src/sandbox/wasmer_backend.rs

core-backend/common/src/funcs.rs

gshep · 2023-08-07T16:33:38Z

from benchmarks 88 - https://github.com/gear-tech/gear/actions/runs/5784482295

Comparison table for Gear runtime

| name                                    | master   | gshep/issue-2920 | diff    |
|-----------------------------------------|----------|------------------|---------|
| alloc                                   | 7.0 µs   | 6.3 µs           | +11.54% |
| alloc_per_page                          | 388.2 ns | 386.6 ns         | +0.41%  |
| free                                    | 1.5 µs   | 787.6 ns         | +91.63% |
| gr_reserve_gas                          | 3.6 µs   | 2.7 µs           | +33.68% |
| gr_unreserve_gas                        | 4.0 µs   | 3.3 µs           | +22.68% |
| gr_system_reserve_gas                   | 2.3 µs   | 1.6 µs           | +49.53% |
| gr_gas_available                        | 2.3 µs   | 1.4 µs           | +58.62% |
| gr_message_id                           | 2.3 µs   | 1.4 µs           | +59.19% |
| gr_pay_program_rent                     | 44.0 ns  | 31.8 ns          | +38.51% |
| gr_program_id                           | 2.3 µs   | 1.4 µs           | +58.56% |
| gr_source                               | 2.3 µs   | 1.4 µs           | +58.11% |
| gr_value                                | 2.3 µs   | 1.4 µs           | +61.46% |
| gr_value_available                      | 2.3 µs   | 1.4 µs           | +61.98% |
| gr_size                                 | 2.3 µs   | 1.5 µs           | +54.83% |
| gr_read                                 | 3.3 µs   | 2.4 µs           | +36.28% |
| gr_read_per_byte                        | 162 ps   | 163 ps           | -0.61%  |
| gr_block_height                         | 2.3 µs   | 1.4 µs           | +57.79% |
| gr_block_timestamp                      | 2.3 µs   | 1.5 µs           | +55.48% |
| gr_random                               | 3.1 µs   | 2.4 µs           | +31.57% |
| gr_reply_deposit                        | 8.5 µs   | 6.9 µs           | +23.42% |
| gr_send                                 | 4.9 µs   | 3.9 µs           | +26.25% |
| gr_send_per_byte                        | 269 ps   | 257 ps           | +4.67%  |
| gr_send_wgas                            | 5.0 µs   | 3.9 µs           | +26.79% |
| gr_send_wgas_per_byte                   | 261 ps   | 258 ps           | +1.16%  |
| gr_send_init                            | 2.4 µs   | 1.6 µs           | +51.75% |
| gr_send_push                            | 3.6 µs   | 2.7 µs           | +30.74% |
| gr_send_push_per_byte                   | 382 ps   | 377 ps           | +1.33%  |
| gr_send_commit                          | 4.2 µs   | 3.4 µs           | +24.51% |
| gr_send_commit_wgas                     | 4.4 µs   | 3.4 µs           | +28.12% |
| gr_reservation_send                     | 5.1 µs   | 4.0 µs           | +25.72% |
| gr_reservation_send_per_byte            | 263 ps   | 260 ps           | +1.15%  |
| gr_reservation_send_commit              | 4.5 µs   | 3.6 µs           | +25.26% |
| gr_reply_commit                         | 25.9 µs  | 20.3 µs          | +27.46% |
| gr_reply_commit_wgas                    | 25.9 µs  | 19.2 µs          | +34.58% |
| gr_reservation_reply                    | 13.3 µs  | 9.7 µs           | +37.23% |
| gr_reservation_reply_per_byte           | 433.8 ns | 429.1 ns         | +1.09%  |
| gr_reservation_reply_commit             | 7.7 µs   | 9.7 µs           | -20.57% |
| gr_reply_push                           | 3.4 µs   | 2.5 µs           | +37.14% |
| gr_reply                                | 20.2 µs  | 19.8 µs          | +1.81%  |
| gr_reply_per_byte                       | 417 ps   | 417 ps           | +0.00%  |
| gr_reply_wgas                           | 24.2 µs  | 17.4 µs          | +38.84% |
| gr_reply_wgas_per_byte                  | 416 ps   | 418 ps           | -0.48%  |
| gr_reply_push_per_byte                  | 646 ps   | 646 ps           | +0.00%  |
| gr_reply_to                             | 2.3 µs   | 1.5 µs           | +58.15% |
| gr_signal_from                          | 2.3 µs   | 1.5 µs           | +59.30% |
| gr_reply_input                          | 7.0 µs   | 24.9 µs          | -71.70% |
| gr_reply_input_wgas                     | 43.8 µs  | 51.2 µs          | -14.40% |
| gr_reply_push_input                     | 2.5 µs   | 1.7 µs           | +46.89% |
| gr_reply_push_input_per_byte            | 160 ps   | 150 ps           | +6.67%  |
| gr_send_input                           | 5.1 µs   | 4.1 µs           | +25.15% |
| gr_send_input_wgas                      | 5.2 µs   | 4.1 µs           | +25.06% |
| gr_send_push_input                      | 2.7 µs   | 2.0 µs           | +34.58% |
| gr_send_push_input_per_byte             | 168 ps   | 166 ps           | +1.20%  |
| gr_debug                                | 2.5 µs   | 1.7 µs           | +42.20% |
| gr_debug_per_byte                       | 318 ps   | 311 ps           | +2.25%  |
| gr_reply_code                           | 2.3 µs   | 1.4 µs           | +62.73% |
| gr_exit                                 | 27.2 µs  | 27.3 µs          | -0.49%  |
| gr_leave                                | 14.6 µs  | 16.4 µs          | -10.71% |
| gr_wait                                 | 15.5 µs  | 13.3 µs          | +15.93% |
| gr_wait_for                             | 14.1 µs  | 14.5 µs          | -2.31%  |
| gr_wait_up_to                           | 13.4 µs  | 11.4 µs          | +17.21% |
| gr_wake                                 | 3.5 µs   | 2.6 µs           | +35.20% |
| gr_create_program                       | 5.9 µs   | 5.0 µs           | +19.26% |
| gr_create_program_payload_per_byte      | 99 ps    | 83 ps            | +19.28% |
| gr_create_program_salt_per_byte         | 2.2 ns   | 2.2 ns           | +1.94%  |
| gr_create_program_wgas                  | 6.0 µs   | 5.0 µs           | +18.77% |
| gr_create_program_wgas_payload_per_byte | 97 ps    | 96 ps            | +1.04%  |
| gr_create_program_wgas_salt_per_byte    | 2.2 ns   | 2.2 ns           | +1.52%  |

sandbox/host/src/sandbox/wasmer_backend.rs

core-backend/wasmi/src/funcs_tree.rs

runtime/vara/src/weights/frame_system.rs

sandbox/env/src/lib.rs

sandbox/host/src/sandbox/wasmer_backend.rs

sandbox/env/src/lib.rs

breathx

Awesome!

sandbox/env/src/lib.rs

sandbox/host/src/sandbox/wasmer_backend.rs

done

1dd2815

gshep added A0-pleasereview PR is ready to be reviewed by the team D2-node Gear Node D7-performance Increase our node/runtime/programs execution work performance labels Jul 20, 2023

gshep self-assigned this Jul 20, 2023

gshep added 6 commits July 20, 2023 22:15

revert signatures for wasmi backend

4faa88f

don't create RuntimeError instantly

35ea3be

preallocate buffer for scale-encoding WasmReturnValue

465747f

remove InstanceGlobals

d4a45e6

make fmt

b46cb8b

fix misprint in funcs codegeneration

b5afb5f

gshep changed the title ~~Draft: refactor(sandbox-host): Process wasm-globals on wasmer side~~ refactor(sandbox-host): Process wasm-globals on wasmer side Jul 23, 2023

gshep requested review from breathx, grishasobol and ukint-vs July 23, 2023 14:04

gshep added 2 commits August 7, 2023 15:09

Merge remote-tracking branch 'origin/master'

379f707

Conflicts: core-backend/common/src/funcs.rs core-backend/sandbox/src/env.rs core-backend/sandbox/src/runtime.rs core-backend/wasmi/src/runtime.rs

make fmt & clippy

9943695