Skip to content

[codex] Fix HNSW delete/reinsert rebuild#142

Merged
joaoh82 merged 1 commit into
mainfrom
fix/hnsw-delete-reinsert
May 25, 2026
Merged

[codex] Fix HNSW delete/reinsert rebuild#142
joaoh82 merged 1 commit into
mainfrom
fix/hnsw-delete-reinsert

Conversation

@joaoh82
Copy link
Copy Markdown
Owner

@joaoh82 joaoh82 commented May 25, 2026

Summary

Fixes SQLR-8: HNSW indexes could panic after rows were deleted and new vectors were inserted within the same open connection.

Root Cause

DELETE / vector-column UPDATE marked HNSW indexes as needs_rebuild, but only the pager save path rebuilt them. In-memory INSERT still appended into the stale graph. Deleted rowids could remain in neighbor lists, and the HNSW get_vec closure returned an empty vector for those stale ids, eventually causing DistanceMetric::compute to assert or index into a zero-length slice.

Changes

  • Rebuild dirty HNSW indexes from surviving rows before the next INSERT on an indexed vector column.
  • Reuse the same rebuild helper from the pager save path so save/COMMIT behavior stays consistent.
  • Change HNSW distance/search/insert paths to return typed SQLRiteErrors for empty or mismatched vector dimensions instead of panicking.
  • Add a regression test for DELETE -> INSERT -> KNN query in one connection.
  • Remove the examples/nodejs-notes db.reopen() workaround and the known-limitation README entry.
  • Update docs/sql-engine.md with the current dirty-index lifecycle.

Validation

  • Original REPL reproducer now completes and returns document_id = 2.
  • cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets
  • cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks
  • cargo test hnsw --lib
  • npm test in examples/nodejs-notes
  • cargo fmt --all -- --check
  • cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets (passes with existing warnings)
  • cargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps (passes with existing rustdoc warnings)

Deploy Plan

Because this is a process-aborting bug in the engine and affects SDK consumers, ship as a patch release.

  1. Merge this PR after CI is green.
  2. Cut a patch version with scripts/bump-version.sh 0.10.1.
  3. Run the release validation matrix from AGENTS.md, including workspace build/test/clippy/doc and examples/nodejs-notes npm test.
  4. Publish release artifacts in dependency order: engine crate, FFI/SDK packages, MCP binary/package, then example docs/site references if applicable.
  5. For examples/nodejs-notes, verify the refresh flow no longer performs a close/reopen hop and announce that consumers can remove their own workaround.
  6. Add a short release note: "Fix HNSW panic after DELETE followed by INSERT in the same connection; HNSW now rebuilds dirty in-memory graphs before reuse and reports typed dimension errors."

Notes

Unrelated local dirty files were intentionally left out of this PR: an untracked AGENTS.md and a mode-only change on examples/nodejs-notes/bin/sqlrite-notes.mjs.

@vercel
Copy link
Copy Markdown

vercel Bot commented May 25, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
rust-sqlite Ready Ready Preview, Comment May 25, 2026 7:48am

Request Review

@joaoh82 joaoh82 marked this pull request as ready for review May 25, 2026 08:04
@joaoh82 joaoh82 merged commit c9327cc into main May 25, 2026
18 checks passed
@joaoh82 joaoh82 mentioned this pull request May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant