Skip to content

Rework Vector Set deletion and eviction#1800

Merged
kevin-montrose merged 20 commits into
mainfrom
users/kmontrose/vectorSetEvictionModernization
May 19, 2026
Merged

Rework Vector Set deletion and eviction#1800
kevin-montrose merged 20 commits into
mainfrom
users/kmontrose/vectorSetEvictionModernization

Conversation

@kevin-montrose
Copy link
Copy Markdown
Contributor

@kevin-montrose kevin-montrose commented May 13, 2026

Rework Vector Set deletion and eviction to use GarnetRecordTriggers.

This matches how Range Indexes work, removes the BADSTATE possibility, gets a GUID comparison out of the hot path, and lays some groundwork for lots of Vector Sets.

Concretely:

  • Switches Vector Set deletion detection to GarnetRecordTriggers.OnDispose, getting rid of all the cancelled Tsavorite op stuff
  • Blocks on marking a context needs cleanup, removing all the partial deletion tracking logic
  • Removes the GUID stuff for detecting index recreation, instead using the OnEvict and OnDiskRead callbacks in the same manner as Ranged Indexes
  • Fixes RecordType handling during copy updates, fixing expiration

TODO:

  • I'm not convince this approach cleanups up element data if we crash before marking - need to validate that
  • Tests for expiration
  • Update vector-sets.md with final delete design

@kevin-montrose kevin-montrose marked this pull request as ready for review May 14, 2026 21:24
Copilot AI review requested due to automatic review settings May 14, 2026 21:24
@kevin-montrose kevin-montrose requested a review from badrishc May 18, 2026 20:41
Comment thread libs/server/Storage/Functions/GarnetRecordTriggers.cs Outdated
@kevin-montrose kevin-montrose requested a review from badrishc May 19, 2026 16:04
@kevin-montrose kevin-montrose merged commit a847079 into main May 19, 2026
368 of 369 checks passed
@kevin-montrose kevin-montrose deleted the users/kmontrose/vectorSetEvictionModernization branch May 19, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants