TiDB in WebAssembly WASI

[syrusakbary]

Feature Request

Describe the feature you'd like:
Following #13069, I think it would be a great idea to compile TiDB to WebAssembly WASI.

This way, there can be a standalone TiDB wasm binary published in WAPM and available in any platform/OS and even online with the WebAssembly shell, so anyone can play with it easily.

Describe alternatives you've considered:
This should be feasible by using the go/wasi implementation.

Teachability, Documentation, Adoption, Migration Strategy:
If this is something that you are interested in, we can help on the implementation :)

[lucklove]

That's so cool! In fact, we are very eager to do this, but we notice that it seems golang/wasi is still working in progress, I'm not sure if it's possible at the present stage.
We want to have a try, it will be very nice if you can help.

[syrusakbary]

After a few trials I got tidb compiled to WASI:

main.wasm.zip

However, the file is so large I've been unable to run it with any runtime 😅

[lucklove]

@syrusakbary It seems that we can only run it in PC's browser now, the phone will report out of execution memory:

Maybe we can reduce the size by remove some useless functions in tidb when we compile to wasm

[siddontang]

maybe we need to fix #13322 at first, to reduce the binary size.

[lucklove]

@syrusakbary I try to write a hello world with golang, and run it with wasmer, but failed, I'm not sure if something is wrong.

[lucklove]

@syrusakbary finally I make it work by a modified wasmer:

I modify one line code:

I know it's just a workaround, instead of the best way to solve this, but I hope this can help to find the root cause.

You can use this code for testing

[syrusakbary]

Love it! Can you upload the wasm generated file somewhere?

@nlewycky

[syrusakbary]

Also, can you try to run it with wasmer-js and see what's the result? (wasmer-js run main.wasm).

Note: you can install wasmer-js with npm install -g @wasmer/cli;

[syrusakbary]

We debugged this internally, and it might be caused by go/wasi generating an invalid WebAssembly file. Depending on your results on the previous questions we will be able to validate this assumption.

What can be causing the issue? A data element has an address that overflow the end of the memory. If that's the case, this could be easily fixed on the go/wasi side (on the compiler from go to WebAssembly WASI).

[lucklove]

The wasm generated file: main.wasm.zip
The result of wasmer-js run main.wasm(the same error with wasmer):

@syrusakbary PTAL

[lucklove]

@syrusakbary I have modified go/wasi and I can run tidb with wasmer(without modify). However, I can't run it with wapm neither in the browser nor in the local shell.
The WebAssembly shell log:

And the wasm file:
main.wasm.zip

[syrusakbary]

We just fixed the issue, and now tidb runs properly on the WebAssembly shell 🎉

Note: we are going to release a new version of wasmer soon that will use the singlepass backend automatically when the file size is very big (solving the issue of running tidb with wapm directly).

https://webassembly.sh/?run-command=tidb

[ngaut]

Wow, amazing job. Thank you @syrusakbary

[lucklove]

@syrusakbary What version the WebAssembly shell will use? There are some bugs in tidb's wasm result and I fixed them, but it seems that the WebAssembly just use an older version in most time (although sometimes it uses newest one)

[syrusakbary]

Perhaps the module is being cached?
Adding @torch2424 to the thread to see what we can do in our side.

[torch2424]

@lucklove Hello! 😄

So If I could ask, how are you adding the Wasm Module? Are you dragging it in? Or installing it from WAPM?

Thanks! 😄 👍

[lucklove]

@torch2424 I just open WebAssembly Shell and input "tidb" (I think it's installed from WAPM). I expect it uses the newest version, however, it uses old ones.
I also try wapm install tidb@version, it seems it downloaded the specified version, however, when I typed "tidb", it downloads old versions again

[lucklove]

Update: I found that it seems webassembly shell store previous downloads in somewhere in the browser so the next time the user input the same command, it doesn't request the network. After clearing cache, it works.

[torch2424]

@lucklove

Ah yep! Sorry for not responding earlier, currently on a work trip!

Ah yep, that's a bug, I'll go ahead and open an issue for this, thank you! 😄 👍

Edit: Opened this issue: wasmerio/webassembly.sh#69

[lucklove]

Closing this since TiDB can work well with WASI now.