feat(engine): Phase 11.1 multi-connection foundation (SQLR-22)#122
Merged
feat(engine): Phase 11.1 multi-connection foundation (SQLR-22)#122
Conversation
First slice of Phase 11 (concurrent writes via MVCC + BEGIN CONCURRENT). Refactors Connection from owning Database by value to holding Arc<Mutex<Database>>, so multiple Connection handles can address the same engine state from inside one process. Connection::connect() mints a sibling handle that shares the backing Database. Connection is now Send + Sync and can be moved across threads without an outer Mutex<Connection>. The pager's existing process-level flock and per-database mutex still serialize commits; true multi-writer throughput on disjoint rows lands with BEGIN CONCURRENT in 11.4. Connection::database() / database_mut() return MutexGuard<'_, Database>; internal call sites in src/ask, sqlrite-mcp tools, and the Python/Node/WASM SDK shims bind the guard to a local first. Six new tests in src/connection.rs: - connect_shares_underlying_database - connect_shares_file_backed_database - handle_count_reflects_live_handles - threaded_writers_serialize_cleanly (8 threads × 25 INSERTs) - prep_cache_is_per_handle - connection_is_send_and_sync (compile-time Send+Sync check) Docs sweep: roadmap gets a Phase 11 section linking to concurrent-writes-plan.md (the plan-doc internally numbers sub-phases as "Phase 10.x" — that's its working title from before Phase 10 = benchmarks shipped). New design-decisions entry 12a. embedding.md gains a "Sharing one database across threads" section. No file-format change. 579/579 workspace tests pass. No new clippy warnings. fmt clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of Phase 11 — concurrent writes via MVCC +
BEGIN CONCURRENT(SQLR-22). RefactorsConnectionfrom owningDatabaseby value to holdingArc<Mutex<Database>>, so multipleConnectionhandles can address the same engine state from inside one process.Connection::connect()mints a sibling handle that shares the backingDatabase.Connection: Send + Sync.Connection::database()/database_mut()(already#[doc(hidden)]) now returnMutexGuard<'_, Database>. Internal call sites insrc/ask/, the sevensqlrite-mcptools that read the database, and the Python / Node / WASM SDK shims bind the guard to a local first.src/connection.rs: in-memory + file-backed sibling sharing, handle counting, 8-thread × 25-INSERT contention, per-handle prep cache, compile-timeSend + Synccheck.Scope honesty
Phase 11.1 is capability, not throughput. Every commit still serializes through the per-database
Mutexand the existing pagerflock. True multi-writer throughput on disjoint rows lands withBEGIN CONCURRENTin 11.4 — see the plan doc for the sequenced rest of Phase 11.The plan doc internally numbers sub-phases as "Phase 10.x" — that's its working title from before Phase 10 (benchmarks) shipped. Roadmap entries say Phase 11.x; code comments and tests track the plan-doc numbering for continuity.
Test plan
cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— cleancargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks— 579/579 pass (32/32 connection tests, including 6 new)cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— no new warningscargo fmt --all -- --check— cleancargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps— no new warnings on changed files.sqlritefiles written before this PR open identicallyFollowups (next sub-phases, separate PRs)
MvccClock+ActiveTxRegistry; persist clock high-water in WAL header (v1 → v2)MvStoreskeleton +PRAGMA journal_mode = mvccsnapshot-isolated readsBEGIN CONCURRENTwrites + commit-time validation (the meat)Busy/BusySnapshot; new bench workloaddocs/concurrent-writes.md🤖 Generated with Claude Code