store: remove sqlite table size analysis by bigeez · Pull Request #2028 · 0xMiden/node

bigeez · 2026-05-01T11:29:22Z

Closes #1964

Summary

Removes the analyze_table_sizes periodic background task and its supporting methods.

The task ran on a 5-minute tokio::time::interval in spawn_grpc_servers and issued two queries against SQLite's dbstat virtual table (a per-page scan) plus a pragma_page_count()/pragma_page_size() query. Under write contention the scan held a read lock long enough to block concurrent writers for 30s+, making it a net negative for the performance it was meant to observe.

Changes

server/mod.rs — removed the spawned interval task; cleaned up now-unused Duration import and updated stale doc comments
db/mod.rs — removed analyze_table_sizes method; cleaned up now-unused QueryableByName import
state/mod.rs — removed analyze_table_sizes wrapper

Disk usage can be inspected on demand via SQLite snapshots or by monitoring the data-directory at the filesystem level (miden-store.sqlite3, blocks/, proof files).

Test plan

cargo check -p miden-node-store passes
No remaining references to analyze_table_sizes, dbstat, or pragma_page in the store crate

The periodic `analyze_table_sizes` task ran every 5 minutes and issued two queries against the `dbstat` virtual table, which performs a full page-by-page scan of the database. Under write contention this scan could block concurrent writers for 30s+, far outweighing any telemetry benefit. Remove the spawned interval task from `server/mod.rs`, and delete `analyze_table_sizes` from `Db` and `State`. Disk usage can be inspected on demand by querying SQLite snapshots or monitoring the data-directory at the filesystem level. Closes 0xMiden#1964

Replace the removed dbstat-based analysis with a 5-minute interval task that reads on-disk byte sizes using std::fs::metadata and std::fs::read_dir. No SQLite connections are opened, so there is zero read-lock contention with concurrent writers. Tracked paths: - miden-store.sqlite3 (main database) - miden-store.sqlite3-wal and -shm (SQLite WAL sidecars) - blocks/ (block store, recursive) - accounttree/ and nullifiertree/ (RocksDB trees, rocksdb feature only)

Mirko-von-Leipzig · 2026-05-02T07:06:59Z

+        #[cfg(feature = "rocksdb")]
+        account_tree: dir_size_bytes(&data_dir.join("accounttree")),
+        #[cfg(feature = "rocksdb")]
+        nullifier_tree: dir_size_bytes(&data_dir.join("nullifiertree")),


@kkovaacs do we need to explore the directory for rocksdb, or can we be more specific like with sqlite?

- Use `Duration::from_mins(5)` (stabilised in Rust 1.91) - Replace `info!`/`warn!` events with `info_span!` and OTel span attributes - Use `db.*` attribute naming to align with OTel conventions - Drop `append_path_str` helper in favour of direct `data_dir.join(...)` calls - Add `ToValue for u64` to support disk-size span attributes

- Replace recursive walk with iterative loop to avoid stack overflow on deeply nested directories - Use match-with-guards in loop body instead of nested if-let

Maintainer prefers the original if-let style over match-with-guards in the context of the iterative walk.

Mirko-von-Leipzig

Thank you! Lets wait for @kkovaacs to also TAL.

Mirko-von-Leipzig requested review from Mirko-von-Leipzig and kkovaacs May 1, 2026 13:36

Mirko-von-Leipzig reviewed May 1, 2026

View reviewed changes

Comment thread crates/store/src/db/mod.rs