Skip to content

Fix Vector Set Index key corruption#1845

Merged
kevin-montrose merged 3 commits into
mainfrom
users/kmontrose/vectorSetIndexManagementFixes
Jun 2, 2026
Merged

Fix Vector Set Index key corruption#1845
kevin-montrose merged 3 commits into
mainfrom
users/kmontrose/vectorSetIndexManagementFixes

Conversation

@kevin-montrose
Copy link
Copy Markdown
Contributor

As part of moving to Store V2, VADD replication writes were moved into the default namespace. This introduced a subtle corruption issue.

In Store V1, since nothing of value was ever stored in the replication namespace it didn't matter if we were doing an update in place or a copy update. In Store V2 the actual index is stored in the same key, which all works fine until we need to copy the index into the mutable part of the log (ie. do a copy update). Since there's no code for that case in V1, we "copy" an empty value and the next read operation to touch that index gets a corrupted index.

This fixes that by properly copying the index when needed.

… CU now that we don't hide it in a separate namespace, so prevent it from clobbering the index by actually copying the index value over
Copilot AI review requested due to automatic review settings June 1, 2026 20:47
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a Store V2 regression where Vector Set replication writes (VADD in the default namespace) can trigger a Tsavorite copy-update (CU) that allocates an empty value, corrupting the vector index key on subsequent reads.

Changes:

  • Ensure CU sizing accounts for the vector index payload when replaying VADD replication writes.
  • Preserve the existing vector index bytes during CU for VADD replication no-op writes (and related CU flows).
  • Remove the now-unused DeleteAfterDropArg constant and re-base arg numbering accordingly.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
libs/server/Storage/Functions/MainStore/VarLenInputMethods.cs Adjusts RMW modified-field sizing for Vector Set operations so CU can reserve enough value space.
libs/server/Storage/Functions/MainStore/RMWMethods.cs Updates CopyUpdater logic to preserve index bytes during CU for synthetic Vector Set replication writes.
libs/server/Resp/Vector/VectorManager.cs Updates internal arg constants (removing the deleted Drop-related arg).

Comment thread libs/server/Storage/Functions/MainStore/VarLenInputMethods.cs
Comment thread libs/server/Storage/Functions/MainStore/RMWMethods.cs Outdated
Comment thread libs/server/Storage/Functions/MainStore/RMWMethods.cs Outdated
@kevin-montrose kevin-montrose merged commit b315b32 into main Jun 2, 2026
137 checks passed
@kevin-montrose kevin-montrose deleted the users/kmontrose/vectorSetIndexManagementFixes branch June 2, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants