fix(store): make LanceStore::update atomic via merge_insert

[doctatortot]

Summary

LanceStore::update() can silently lose a memory if the operation is interrupted partway through. It replaces a row with two separate, independently-committed operations — a delete followed by an add — and if execution stops between them, the row is deleted but never re-inserted.

Root cause

// store/lancedb.rs — update()
table.delete(&format!("id = '{}'", escape_sql(&mem.id))).await?;   // commit #1
// ... build batch ...
table.add(reader).execute().await?;                                // commit #2

These are two distinct transactions. If the future is dropped at the second .await — which happens whenever the caller's request is cancelled (e.g. an HTTP client disconnects/aborts and the axum handler future is dropped) — the delete has already committed but the add never runs. The row is gone with no replacement.

soft_delete() goes through the same update() path, so DELETE /v1/memories/{id} and batch_soft_delete are exposed too: a cancelled delete can drop the row entirely instead of tombstoning it (state = 'deleted').

When it bites

It's easy to miss because the happy path works and create() (a single add) is already atomic. It shows up under interruption, and is most likely when an update is slow enough that a client times out and aborts mid-flight — e.g. a content-changing update that re-embeds a large document on a busy/slow host. The result is a memory that "vanished" with no error on the server side.

Fix

Replace the delete-then-add with a single atomic merge_insert upsert keyed on id:

let mut merge = table.merge_insert(&["id"]);
merge.when_matched_update_all(None);
merge.when_not_matched_insert_all();
merge.execute(reader).await?;

This commits as one transaction, so an interrupted update either fully applies or not at all — it can never leave the row deleted-without-replacement. Behavior is otherwise preserved: the version bump + updated_at are still computed before the write, and updating an id that doesn't exist still inserts it (the old delete-no-op + add did the same; when_not_matched_insert_all keeps that). Rows are keyed by unique ids, so the single-match update path applies.

Tests

• test_update_replaces_row_atomically — update replaces a row in place: exactly one copy remains (no duplicate, no loss), content + version updated.
• test_update_upserts_when_row_absent — update inserts when the row is absent.
• Full suite green (cargo test, 381 passed). cargo fmt --check and clippy clean for the change.

Notes

merge_insert is available in the pinned lancedb (0.27). No schema or API changes; purely an internal durability fix in update().

[yhyyz]

Merged and deployed — great catch @doctatortot! The delete-then-add race condition was a real data-loss bug waiting to happen. merge_insert is the correct fix. 👍