Support sql.js for sqlite native support

[khunder]

Does Sidequest.js currently support using sql.js as a SQLite backend?
I am looking for a lightweight, database-agnostic setup that can run without requiring native SQLite bindings such as better-sqlite3 or sqlite3.
Supporting sql.js would make Sidequest.js easier to use in environments where native modules are difficult or impossible to install.

[GiovaniGuizzo]

Hey @khunder.

No, unfortunately, we don't support sql.js. You could try creating a new backend with that driver.

However, I doubt that would fully work. From what I could tell, it's not persisted anywhere and is only in memory right? Moreover, it needs to be distributed somehow. I'm saying this because we spin up workers and other services that need a "shared" DB across multiple threads, not an in-memory db. That's why something like PGlite doesn't work either.

We recently introduced a new feature where you can run everything in a single thread, so you can share memory states. Maybe that would work with a driver like sql.js or PGLite.

[khunder]

Thanks @GiovaniGuizzo , that makes sense.

I’ve already forked the project and prototyped a modified SQLite backend using sql.js, with some help from Claude. I’m currently using it for testing instead of maintaining fake repositories backed by static arrays.

all the 80+ tests are passing, but there is a catch.

While sql.js runs the active database in memory, it can import and export a complete SQLite database file.
My prototype loads the file when the backend starts and writes a new snapshot after committed changes. However, it does not solve the distributed-worker problem, separate processes would hold independent snapshots and could overwrite each other’s changes.

The new single-thread execution mode sounds like a good fit for this use case.
My goal is only a lightweight test/development backend without requiring node-gyp, Python, or native SQLite bindings especially in CI.

The current implementation is still experimental and not production-ready, but I’d be happy to share it if it could be useful as a starting point.

[GiovaniGuizzo]

Interesting. I suggest you try no fork and no threads config. You can find more info on the docs. This will ensure everything will use the same memory space.