New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Another Performance Problem Compared With Other Runtime Tools #759
Comments
Thanks a lot @luxinyi0105 ! The WAMR (JIT mode) also seems to be extremely slow for it being a JIT. Maybe you could also open an issue there? Note that Wasmer, Wasmtime (and most certainly WasmEdge with AoT) are known for performing tons of optimizations to the input Wasm files before and during execution while a Wasm interpreter applies only a few if any. At least, maybe this could explain the huge differences we see here. A 43x slowdown from Wasmtime to Wasm3 is also slower than I usually measure for Wasm3. Usually Wasm3 is 5-20x slower than Wasmtime. We could try to check this by applying Binaryen's The register-machine based |
@luxinyi0105 I have a new suspicion about the performance testings. You mutated the Wasm blobs without applying proper optimizations. This leads to optimizing JIT and AoT Wasm engines to highly outperform non-optimizing interpreter based Wasm runtimes such as Wasm3 and For example, at Parity we optimize our Wasm smart contracts using LLVM, fat LTO and |
Note that this issue might be fixed by the upcoming release of the |
Description
Similar to issue #754, I found another testcase that requires a lot more running time than other runtime tools.
Versions and Environment
Tools version: wasmi_cli 0.31.0
Operating system: Ubuntu 22.04.1
Architecture: x86_64
Explanation of TestCase
The given testcase was simply mutated a wasm file, which was obtained by compiling C program generated with
Csmith
usingEmscripten Compiler
(Emcc).The original C program is
c_file.c
, the compilation results with Emscripten iswasm_file.wasm
, and its wat format iswat_file.wat
.We mutated the wat file to change all
i32/i64.add
toi32/i64.sub
, and change alli32/i64.shl
toi32/i64.rotr
. The result after mutation ismutated_file.wat
, and its wasm format ismutated_file.wasm
.Extra Info
Although it takes different execution times, the running results obtained by different runtime tools are the same:
RuntimeError
due toout of bounds memory access
. The execution time of wasmi and other runtime tools are as follows:wasmer
:28.679 s
wasmtime
:22.521 s
wasmedge with AOT mode
:9.485 s
wasm-micro-runtime with JIT mode
:2 m 55.342 s
wasm3
:15 m 54.901 s
wasmi
:140 m 58.431 s
I'm not sure if the problem in this issue is similar with issue #754, the detail still needs you to confim later. Thanks a lot!
The text was updated successfully, but these errors were encountered: