How to test the WASM backend

[certik]

Original issue: https://gitlab.com/lfortran/lfortran/-/issues/689

Here are several ideas.

1. The way we can test it is by using our existing ./run_tests.py, which checks the hash of the generated output (the wasm binary file), but we will not check it into git (it's not a good idea to have binary files in git). Everything will work (./run_tests.py was designed to allow this), but one disadvantage is that if some MR changes tests results (say we refactor how the wasm backend works a little bit, which we will do almost for sure from time to time), we'll see that the hash changed, and we'll see the new wasm binary files, but it might not be obvious if we didn't break anything. It would not be quite obvious anyway, even if the old binaries were in git. Plus we can always generate them from master.

2. The second idea is to first generate the wasm binary, then convert it to wat, and store the wat files in git. That way any change will be obvious, as they are stored as text. Just like we store the ASR results.

The second option I think is preferable. There are two further ways to do it:

2a. We can first generate the wasm binary, then convert it to wat. We can either use wabt to do it by calling wasm2wat (but then LFortran would always depend on it), or we do it as part of LFortran itself (then we need to link the wabt library), or we write our own code to do that.

2b. We generate wat directly by the backend/assembler. We then need to maintain both ways, it will be slower (so possibly we only want to do it when a macro define is enabled.

I don't know which option is better.

However, we also want to have wasm->x86/arm backends. It seems to me those backends should take the wasm binary as input, they should decode it and then transform into x86 or arm (and other platforms if needed). Being able to start from a wasm binary would allow this backend to be tested completely independently of LFortran, and usable for any wasm binary. Which I think is a good design.

So treating the wasm binary as the "ground truth", as the IR, seems like a good design. The alternative is to have WASM.asdl and generate our own IR that represents the WASM binary, which we can do also, but it seems the wasm binary itself has been design in a way that can be used as our IR directly. All we need is to write a "visitor" that can visit the individual nodes of the wasm binary.

So once we have this visitor (that we need for the wasm->x86/arm backends), we can also use this visitor to print the wasm in textual form. I have noticed that WAT doesn't directly corresponds to the binary: it seems it lumps together the function header with the body. So we can possibly have our own textual representation that directly maps what is in the wasm binary (it seems one has to do further transformations on it to get WAT).

It seems to me the best way forward is to keep the current wasm backend as is, that is, it emits a binary. Then let's write a reader for (any) wasm binary that "visits" every element (it gives you access to all the sections and then iterates over each section). Then we'll write a visitor that emits a textual representation and use it for our tests. And we then also write a different visitor that emits x86 and arm machine codes.

[Shaikh-Ubaid]

Update: We currently have a wasm_to_wat visitor which generates wat from given wasm_bytes. To test the wasm backend we have two ways:

• wasm integration_tests - we generate wasm for the source code and also execute it
• wat tests tested using ./run_tests.py - we generate wat for the source code (here wasm is first generated and is treated as an intermediate output/representation)

[certik]

I think this is fixed, thanks!

[Shaikh-Ubaid]

#636 enables (approximately) all the currently supporting integration_tests for wasm.