Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,12 +113,17 @@ docker-compose up -d

### SQLite Package

- RivetKit SQLite is native-only: VFS and query execution live in `rivetkit-rust/packages/rivetkit-sqlite/`, core owns lifecycle, and NAPI only marshals JS types.
- Depot client owns native SQLite VFS and query execution in `engine/packages/depot-client/`; core owns lifecycle, and NAPI only marshals JS types.
- SQLite VFS direct tests should record workflow compaction wakes through `CompactionSignaler`, not call legacy `compact_default_batch`.
- SQLite VFS correctness tests should use `DirectStorage`; do not reintroduce mock or envoy transport variants for those tests.
- SQLite VFS `xSync` durability depends on depot's `sqlite_commit` reply waiting for the FDB transaction commit.
- SQLite VFS process-global registrations must be owned by a Drop guard so panics unwind through `sqlite3_vfs_unregister`.
- `NativeDatabase::Drop` must bound dirty-page flushes with a short timeout and return after logging if the commit future never resolves.
- Actor2 workflows and envoy actors always use the SQLite v2 storage format; only old actor v1 workflows and pegboard runners use the v1 storage format. ("v2" here refers to the on-disk storage format, not envoy-protocol v2.)
- Native SQLite VFS recent-page preload hints are actor-side Rust state surfaced by `NativeDatabase::snapshot_preload_hints()`; persist and consume them through runtime/envoy wiring, not JS APIs.
- SQLite VFS file handles must enforce their reader or writer role; reader-owned handles fail closed on mutating callbacks.
- Native SQLite single-statement work should route through the native execute path; keep `exec` as the multi-statement compatibility path.
- Pegboard-envoy remote SQL execution should use `rivetkit-sqlite::database::open_database_from_engine` instead of direct `rusqlite` calls so native routing policy stays shared.
- Native SQLite opens should use `depot_client::database::open_database_from_envoy` instead of direct `rusqlite` calls so native routing policy stays shared.
- Pegboard-envoy remote SQL executor caches should use `Arc<OnceCell<NativeDatabaseHandle>>` values so first-use initialization stays lazy and single-flight per `(actor_id, sqlite_generation)`.
- Sent remote SQL requests must fail with `sqlite.remote_indeterminate_result` on WebSocket disconnect; only unsent remote SQL may be sent after reconnect.
- Native SQLite manual transactions keep an idle writer open until autocommit returns; route subsequent work through the writer instead of reader classification.
Expand Down
72 changes: 36 additions & 36 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ members = [
"rivetkit-rust/packages/actor-persist",
"rivetkit-rust/packages/rivetkit-core",
"rivetkit-rust/packages/shared-types",
"rivetkit-rust/packages/rivetkit-sqlite",
"rivetkit-rust/packages/rivetkit-sqlite-types",
"engine/packages/depot-client",
"engine/packages/depot-client-types",
"rivetkit-typescript/packages/rivetkit-napi",
"rivetkit-typescript/packages/rivetkit-wasm"
]
Expand Down Expand Up @@ -558,11 +558,11 @@ members = [
[workspace.dependencies.rivetkit-inspector-protocol]
path = "rivetkit-rust/packages/inspector-protocol"

[workspace.dependencies.rivetkit-sqlite]
path = "rivetkit-rust/packages/rivetkit-sqlite"
[workspace.dependencies.depot-client]
path = "engine/packages/depot-client"

[workspace.dependencies.rivetkit-sqlite-types]
path = "rivetkit-rust/packages/rivetkit-sqlite-types"
[workspace.dependencies.depot-client-types]
path = "engine/packages/depot-client-types"

[workspace.dependencies.rivetkit-core]
path = "rivetkit-rust/packages/rivetkit-core"
Expand Down
29 changes: 6 additions & 23 deletions docs-internal/engine/sqlite-vfs.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,13 @@
# SQLite VFS parity
# SQLite VFS

Rules for the SQLite VFS implementations.
Rules for the SQLite VFS implementation.

## Package boundaries

- RivetKit SQLite is native-only. VFS and query execution live in `rivetkit-rust/packages/rivetkit-sqlite/`, core owns lifecycle, and NAPI only marshals JS types.
- RivetKit TypeScript SQLite is exposed through `@rivetkit/rivetkit-napi`, but runtime behavior stays in `rivetkit-rust/packages/rivetkit-sqlite/` and `rivetkit-core`.
- The Rust KV-backed SQLite implementation lives in `rivetkit-rust/packages/rivetkit-sqlite/src/`. When changing its on-disk or KV layout, update the internal data-channel spec in the same change.

## Native VFS ↔ WASM VFS parity

**The native Rust VFS and the WASM TypeScript VFS must match 1:1.** This includes:

- KV key layout and encoding
- Chunk size
- PRAGMA settings
- VFS callback-to-KV-operation mapping
- Delete/truncate strategy (both must use `deleteRange`)
- Journal mode

When changing any VFS behavior in one implementation, update the other.

- Native: `rivetkit-rust/packages/rivetkit-sqlite/src/vfs.rs`, `kv.rs`
- WASM: `rivetkit-typescript/packages/sqlite-wasm/src/vfs.ts`, `kv.ts`

The native VFS uses the same 4 KiB chunk layout and KV key encoding as the WASM VFS. Data is compatible between backends.
- Depot client owns native SQLite VFS and query execution in `engine/packages/depot-client/`. Core owns lifecycle, and NAPI only marshals JS types.
- RivetKit TypeScript SQLite is exposed through `@rivetkit/rivetkit-napi`, but runtime behavior stays in `engine/packages/depot-client/` and `rivetkit-core`.
- The Rust KV-backed SQLite implementation lives in `engine/packages/depot-client/src/`. When changing its on-disk or KV layout, update the internal data-channel spec in the same change.
- The VFS uses a 4 KiB chunk layout for page storage. PRAGMAs are pinned at open: `journal_mode = DELETE`, `locking_mode = EXCLUSIVE`, `auto_vacuum = NONE`. Source: `engine/packages/depot-client/src/vfs.rs`.

## VFS implementation notes

Expand Down
2 changes: 1 addition & 1 deletion docs-internal/engine/sqlite/vfs-brief.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ The VFS does not write local SQLite database files. Local files would break the

- [SQLite VFS](../sqlite-vfs.md)
- [Depot crash course](../depot.md)
- VFS source: `rivetkit-rust/packages/rivetkit-sqlite/src/`
- VFS source: `engine/packages/depot-client/src/`
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
[package]
name = "rivetkit-sqlite-types"
name = "depot-client-types"
version.workspace = true
authors.workspace = true
license.workspace = true
edition.workspace = true
workspace = "../../../"
description = "Shared SQLite execution types for RivetKit"
description = "Shared SQLite execution types for depot client"

[lib]
crate-type = ["lib"]
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! Shared SQLite execution types for local and remote RivetKit backends.
//! Shared SQLite execution types for local and remote depot client backends.

#[derive(Clone, Debug, PartialEq)]
pub enum BindParam {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
[package]
name = "rivetkit-sqlite"
name = "depot-client"
version.workspace = true
edition.workspace = true
authors.workspace = true
license.workspace = true
workspace = "../../../"
description = "Native SQLite VFS for RivetKit backed by depot"
description = "Depot-backed native SQLite VFS client"

[lib]
crate-type = ["lib"]
Expand All @@ -18,7 +18,7 @@ tokio.workspace = true
tracing.workspace = true
getrandom = "0.2"
rivet-envoy-protocol.workspace = true
rivetkit-sqlite-types.workspace = true
depot-client-types.workspace = true
moka = { version = "0.12", default-features = false, features = ["sync"] }
parking_lot.workspace = true

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! SQLite library for RivetKit.
//! Depot-backed SQLite client.
//!
//! Provides the native SQLite VFS used by RivetKit actors.
//! Provides the native SQLite VFS used by Rivet actors.
//!
//! This is a pure Rust library. N-API bindings and transport clients
//! live in separate crates that compose this one.
Expand All @@ -26,7 +26,7 @@ pub mod optimization_flags;
/// SQLite query execution helpers.
pub mod query;

/// Custom SQLite VFS for actor-side sqlite-storage transport.
pub use depot_client_types as types;

pub use rivetkit_sqlite_types as types;
/// Custom SQLite VFS for actor-side depot transport.
pub mod vfs;
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ use libsqlite3_sys::{
sqlite3_column_type, sqlite3_errmsg, sqlite3_finalize, sqlite3_last_insert_rowid,
sqlite3_prepare_v2, sqlite3_set_authorizer, sqlite3_step, sqlite3_stmt_readonly,
};
pub use rivetkit_sqlite_types::{
pub use depot_client_types::{
BindParam, ColumnValue, ExecResult, ExecuteResult, ExecuteRoute, QueryResult,
};

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Custom SQLite VFS backed by KV operations over the KV channel.
//!
//! This crate now owns the KV-backed SQLite behavior used by `rivetkit-napi`.
//! This crate owns the KV-backed SQLite behavior used by `rivetkit-napi`.

use std::collections::{BTreeMap, HashMap, HashSet, VecDeque};
use std::ffi::{CStr, CString, c_char, c_int, c_void};
Expand Down
1 change: 1 addition & 0 deletions engine/packages/depot-client/tests/inline/fault/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -893,6 +893,7 @@ fn open_fault_database(
actor_id.to_string(),
handle.clone(),
config,
None,
)
.map_err(anyhow::Error::msg)?;

Expand Down
Loading
Loading