Feat: Add WasmEdge as a UDF runtime

[juntao]

Happy new year! And congratulations on the successful incorporation of Wasm UDFs into the libsql project! As we had discussed in the PR, we would like to explore adding WasmEdge as an option for UDFs in libsql. The rationales are as follows.

• WasmEdge is one of the smallest and fastest Wasm runtimes.
• WasmEdge supports rich APIs in the Wasm runtime beyond the WASI spec, such as async sockets. That enables more complex UDFs.
• Docker starts to bundle and ship WasmEdge in every copy of Docker Desktop from 4.15. That allows over 10M Docker developers to create and test Wasm functions using WasmEdge APIs.
• Fedora and Red Hat EPEL ship with WasmEdge. They could further reduce the size of libsql’s install package.

WasmEdge supports a comprehensive Rust SDK and C SDK to make integration easier. I think we could provide a compile time or runtime flag to switch between supported Wasm runtimes. Please let me know what you think! Thanks.

[psarna]

Thanks! I even experimented a little bit with WasmEdge API back in November - https://github.com/psarna/libsql/commits/wasmedge_poc. This branch is very much not up-to-date, but once I finish some other priority items, I'll get back to it

[juntao]

Please let us know how we can be of assistance. I think the integration still needs to be done by the libsql team. But if there is any questions / feature requests for the WasmEdge SDK, we would be more than happy to help.

And after the integration, we could help developing some use cases. 🤝

[psarna]

Update: I was preempted from UDF for quite a while, but I spent some time this week to recover and rebase my early patch for WasmEdge support. I have ints and floats working as well as a functional CI (which does not pass tests yet, expected). I'm also on vacation next week, but right after that I'll try to deliver a working WasmEdge support. Then we can figure out a nice demo or use case. Thanks!

[juntao]

That sounds great! Look forward to seeing it running!

[psarna]

@juntao and see you at wasmio.tech! (:

[psarna]

#131 now has all the types working, but it's still not mergeable - the test suite needs to be updated in order to cover it, and we should perhaps add some automation for downloading libwasmedge if somebody decides to use it.

(I'm on archlinux where some good person already created an AUR package for wasmedge, but rest of the distros are not covered to the best of my knowledge)

[juntao]

WasmEdge has official packages on Fedora and Red Hat EPEL

https://packages.fedoraproject.org/pkgs/wasmedge/wasmedge/index.html

🤝

[psarna]

Perfect!

[juntao]

perhaps add some automation for downloading libwasmedge if somebody decides to use it

I think the script can just get the slim version of WasmEdge runtime from our release page like this:

https://github.com/WasmEdge/WasmEdge/releases/download/0.11.2/WasmEdge-slim-runtime-0.11.2-manylinux2014_x86_64.tar.gz

The lib64/libwasmedge.so linked file is the one we need for Linux. Both x86_64 and aarch64 versions are available on the release asset page.

[psarna]

Yeah, that's precisely what I've done in the CI script that runs on Ubuntu 👌