Define libSQL<->Wasm ABI for type translation #16
Comments
4 tasks
|
After more consideration - Blobs would likely either require one extra layer of indirection, passing a single blob parameter as two WebAssembly parameters - pointer and size, or encoding its size in the first 4 bytes of data. Extra layer of indirection sounds more in line with the C ABI mentioned above. Then, a blob would be passed as a pointer to a structure |
This was referenced Oct 7, 2022
penberg
added a commit
that referenced
this issue
Nov 21, 2022
…otr Sarna (#45) This series implements a mechanism for registering and running Wasm functions. The current runtime of choice is wasmtime and its libwasmtime.so library with C bindings (but a switch to Rust should be considered, because that's the native language of wasmtime and the only interface which offers all of its features). It operates on a very crude ABI (ref:#16), where ints and doubles are passed to/from WebAssembly as is, and for strings/blobs/null it passes a pointer to a structure: string: [1 byte for type specification][data] blob: [1 byte for type specification][4 bytes of size][data] null: [1 byte for type specification] The way it's implemented now is twofold: There's an internal run_wasm function, capable of running WebAssembly and translating the parameter types from and to the Wasm module A dynamic lookup table, currently a regular SQL table: CREATE TABLE libsql_wasm_func_table(name text PRIMARY KEY, body text). The table can be initialized from C code by calling libsql_try_initialize_wasm_func_table() or from shell by using a .init_wasm_func_table command. After creating and filling the new meta-table, when a function call is used in a statement, e.g. SELECT id, fib(id) FROM t, and function fib is neither built-in nor user-defined, it will be looked up in the table. If found, its body will be assumed to hold valid WebAssembly code, compiled and run. In order to enable WebAssembly integration, run configure with ./configure --enable-wasm-runtime parameter. A few examples WebAssembly-based user-defined functions coded in Rust can be found here: https://github.com/psarna/libsql_bindgen Here's an inline demo for testing purposes, with a WebAssembly fibonacci sequence already compiled from Rust and copied in-place: ``` .init_wasm_func_table CREATE FUNCTION fib LANGUAGE wasm AS ' (module (type (;0;) (func (param i64) (result i64))) (func $fib (type 0) (param i64) (result i64) (local i64) i64.const 0 local.set 1 block ;; label = @1 local.get 0 i64.const 2 i64.lt_u br_if 0 (;@1;) i64.const 0 local.set 1 loop ;; label = @2 local.get 0 i64.const -1 i64.add call $fib local.get 1 i64.add local.set 1 local.get 0 i64.const -2 i64.add local.tee 0 i64.const 1 i64.gt_u br_if 0 (;@2;) end end local.get 0 local.get 1 i64.add) (memory (;0;) 16) (global $__stack_pointer (mut i32) (i32.const 1048576)) (global (;1;) i32 (i32.const 1048576)) (global (;2;) i32 (i32.const 1048576)) (export "memory" (memory 0)) (export "fib" (func $fib))) '; CREATE TABLE IF NOT EXISTS example(id int PRIMARY KEY); INSERT OR REPLACE INTO example(id) VALUES (7); INSERT OR REPLACE INTO example(id) VALUES (8); INSERT OR REPLACE INTO example(id) VALUES (9); SELECT id, fib(id) FROM example; ``` This series also comes with syntactic sugar for registering and deregistering Wasm functions dynamically via SQL: CREATE FUNCTION and DROP FUNCTION: Fixes #18 Fixes #17
penberg
added a commit
to penberg/libsql
that referenced
this issue
Jun 19, 2023
mvcc: add caching rows in the MVCC cursor
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In order to add dev-friendly support for running WebAssembly functions within libSQL, we need to specify the format in which types are passed between the two environments. For instance, when a WebAssembly function is to be called on columns of types
integer,real,text,blob, there needs to be a guideline on how the header of this WebAssembly function should look like.WebAssembly has a narrow set of supported types (https://webassembly.github.io/spec/core/syntax/types.html), and there are a few fundamental questions:
\0or with their size encoded in the prefix?The most reasonable way seems to be to follow another well defined ABI - the C one: https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md#function-signatures
In particular, it specifies that all non-trivial types, like structs, are passed indirectly, via a pointer. I suggest we follow this practice and decide that all parameters of the WebAssembly function are expected to be simply pointers to the
sqlite3_valuestruct. This structure also stores its type information, so the exact type can be inferred and validated within a WebAssembly function.Ref: https://github.com/libsql/libsql/blob/aafe1d5457936a05bca5d312e2fd25f32afeb20b/src/vdbeInt.h#L208-L231
With such a specification, helper libraries (#15) could take care of translating and validating the types, while the users have a clear ABI to follow when implementing user-defined functions which work on libSQL column types.
The text was updated successfully, but these errors were encountered: