Driving Asyncify from Wasm module fails to compile

[RReverser]

Here's the simplest Wasm that tries to import Asyncify functions as per https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html:

(module
  (import "asyncify" "start_unwind" (func $asyncify.start_unwind (param i32)))
  (import "asyncify" "stop_unwind" (func $asyncify.stop_unwind))
  (import "asyncify" "start_rewind" (func $asyncify.start_rewind (param i32)))
  (import "asyncify" "stop_rewind" (func $asyncify.stop_rewind))
  (func $f
    (call $asyncify.stop_unwind)))

Currently compiling this to Wasm and trying to pass through wasm-opt --asyncify this results in:

$ wasm-opt --asyncify test2.wasm
Fatal: Module::getFunction: asyncify_stop_unwind does not exist

This looks like a regression and started occuring after upgrading Binaryen.

Unfortunately, I have no idea which version I had before, and there is no wasm-opt --version, but it should've been ~half a year old, and the issue appears in at least Binaryen 90.

What's weirder is that there is a built-in test for this - https://github.com/WebAssembly/binaryen/blob/master/test/unit/input/asyncify-pure.wat - that should've caught this on CI.

I've decided to check if it still works the way it's called in the test runner and, yes, invoking wasm-opt --asyncify asyncify-pure.wat succeeds.

Next I've realised that I've been compiling my file to Wasm, but this test is passed directly in the test runner.

So I've decided to compile it to Wasm using wat2wasm. This failed because Wat turns out to be malformed:

asyncify-pure.wat:3:4: error: imports must occur before all non-import definitions
  (import "spectest" "print" (func $print (param i32)))
   ^^^^^^
asyncify-pure.wat:4:4: error: imports must occur before all non-import definitions
  (import "asyncify" "start_unwind" (func $asyncify_start_unwind (param i32)))
   ^^^^^^
asyncify-pure.wat:5:4: error: imports must occur before all non-import definitions
  (import "asyncify" "stop_unwind" (func $asyncify_stop_unwind))
   ^^^^^^
asyncify-pure.wat:6:4: error: imports must occur before all non-import definitions
  (import "asyncify" "start_rewind" (func $asyncify_start_rewind (param i32)))
   ^^^^^^
asyncify-pure.wat:7:4: error: imports must occur before all non-import definitions
  (import "asyncify" "stop_rewind" (func $asyncify_stop_rewind))
   ^^^^^^

After fixing it (moving the memory line down after imports), compilation succeeds and now wasm-opt --asyncify on the output fails as well just like on my trimmed down example:

$ wat2wasm asyncify-pure.wat -o asyncify-pure.wasm
$ wasm-opt --asyncify asyncify-pure.wasm
Fatal: Module::getFunction: asyncify_stop_unwind does not exist

At this point I've realised that the assertion might be talking about literal function name, which exists in Wat but probably doesn't get preserved in the generated Wasm.

So I've decided to try and preserve them:

$ wat2wasm asyncify-pure.wat -o asyncify-pure.wasm --debug-names

Surely that should work now...

$ wasm-opt --asyncify asyncify-pure.wasm -o asyncify-pure.out.wasm
unknown name subsection at 365

Well, this gives yet another error - presumably --debug-names preserves some names (probably for locals?) that wasm-opt doesn't recognise.

However, it did produce the output file this time, and it looks correct, so I guess the above was just a warning (which is not very obvious from the output).

Repeating same steps with my original test file still fails with the same error, which confirms the suspicion that wasm-opt tries to find a function by its internal debug name ($asyncify_stop_unwind) and not just by import path.

Summary

All in all, looks like it's not possible to drive Asyncify from compiled WebAssembly at the moment, and only Wat works.

In the course of this adventure it looks like I've encountered couple of independent issues:

• By default wasm-opt accepts syntax that wat2wasm doesn't, and some tests turn out to be malformed. Such inconsistency within same toolchain seems counter-intuititive, but maybe intentional? Not sure. I found --no-check later, but still, defaults matter. Correction: these are different toolchains.
• Some tests are malformed in terms of strict Wat. I think there's value in making them compatible with any tooling, and then using them as both .wat as well as compiled .wasm in the test runner to help catch similar issues.
• wat2wasm generates a name subsection that wasm-opt cannot parse.
• wasm-opt should probably prefix warnings with warning: to make it more obvious that it didn't error out.
• Finally, the main issue - there is a regression in wasm-opt that, apparently, checks for debug names that asyncify functions are imported with and not the import paths themselves, and fails on modules without debug info (e.g. release builds) as well as when debug names don't match the expected ones (e.g. LLVM just emits $base and so Asyncify can't be driven from C / C++ / Rust).

[RReverser]

I suspect the culprit of the primary issue is somewhere in #2404, although I don't understand these changes enough to make a more meaningful guess.

However, downgrading to binaryen 89 seems to work around the issue for now.

[kripken]

By default wasm-opt accepts syntax that wat2wasm doesn't, and some tests turn out to be malformed. Such inconsistency within same toolchain seems counter-intuititive, but maybe intentional? Not sure. I found --no-check later, but still, defaults matter.

Those are different toolchains, though? wasm-opt is from binaryen and wat2wasm from wabt. They actually do have different text formats, too, see notes on text format in https://github.com/WebAssembly/binaryen/#binaryen-ir

wat2wasm generates a name subsection that wasm-opt cannot parse.

Perhaps wabt supports a subsection for global names, or var names, or one of the other extensions that were discussed. That hasn't been a priority for us to implement, but if it would be useful maybe we should.

[kripken]

Oh, although here, the wat differences do seem to be not related to the IR differences. Perhaps they are due to historical changes in the wat format over time. Anyhow, in practice we don't expect wat compatibility between these tools due to the IR issue, so it hasn't been a priority to fix.

[RReverser]

Those are different toolchains, though?

You're right, I mixed them up, sorry. Although I'm still surprised the text format is not strictly compatible between different tools though, but I guess it's not really a binaryen issue then.

[RReverser]

I think this can be closed now; I'm not sure how actionable the WAT mismatch sub-issues are, but the main issue has been resolved.