New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support compiling libSQL to Wasm with virtual WAL #695
Conversation
@psarna could we base that on WAL v2 😄 It'll never get merged if we keep adding stuff on top of v1 🙏 |
ah wait it's actually only VFS |
it's only a VFS stub for now, next week I'll fill all the implementation and then add virtual WAL too. But even regular WAL should work if the VFS is properly implemented, and that will be a major win: we'll have proof that libsql runs in WAL mode when compiled to Wasm. |
my point was more that I will need to port this code with wal v2, which means wal v2 is not getting merged any time soon 😢 |
Why? It isn't in any way intrusive with WAL, I'll just first implement it with what we have, and then port it to walv2 later, but it should be quite trivial (and nonblocking, worst case it breaks for a short while) |
It's far from working, but it already properly calls back from a WebAssembly instance to the host for VFS operations. We also want the same for virtual WAL and we'll have a solid foundation for running libSQL in a Wasm runtime.
Breakthrough: the demo managed to use libsql compiled to Wasm in WAL mode, and store actual data in WAL. Next steps - make it a library, and add virtual WAL support. |
c24c91c
This PR contains a demo which uses Wasmtime runtime to run libSQL in a WebAssembly instance, compiled in WAL mode. Right now it's full of stubs, and a VFS that calls back to the host for each operation. Once these stubs are implemented and we also provide a virtual WAL implementation that calls back to the host, we'll have a solid foundation for running libSQL with replication as a Wasm instance.
Step 1: compile
libsql.wasm
(should be automated with build.rs, I'm just lazy)cd libsql-sqlite3 make wasi
Step 2: run the demo
cd ext/libsql-wasi-demo RUST_LOG=info,libsql_wasi_demo=trace cargo run
Step 3: profit
Virtual WAL is not implemented in this patch, but this demo already successfully used WAL mode from a WebAssembly instance and created a WAL.
Next steps (arbitrary order):