Skip to content

Fix/increment update dir vector store#773

Merged
qin-ctx merged 2 commits intomainfrom
fix/increment_update_dir_vector_store
Mar 19, 2026
Merged

Fix/increment update dir vector store#773
qin-ctx merged 2 commits intomainfrom
fix/increment_update_dir_vector_store

Conversation

@myysy
Copy link
Collaborator

@myysy myysy commented Mar 19, 2026

Description

This PR adds a directory-only L0/L1 vector index migration helper to VikingFS and integrates it into the incremental sync path, ensuring directory summary/overview indexes stay consistent after sync/move operations.

Related Issue

N/A

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Move and implement a private VikingFS._mv_vector_store_l0_l1(old_uri, new_uri) helper to migrate only directory L0/L1 vector index records, with locking and directory validation.
  • Extend VikingVectorIndexBackend.update_uri_mapping and VikingFS._update_vector_store_uris to support levels filtering (so only selected levels are migrated) and apply owner_space restriction for USER contexts.
  • Fix incremental sync flow to correctly await the L0/L1 index migration and guard failures with logging to avoid breaking directory sync.

Contributors

Co-authored-by: @gilangjavier

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

myysy added 2 commits March 19, 2026 18:03
Implement functionality to move L0/L1 vector index entries when directories are relocated. This ensures semantic search indexes remain consistent during file operations. The changes include:
- New _mv_vector_store_l0_l1 method in VikingFS
- Enhanced _update_vector_store_uris with level filtering
- Added tests for the new functionality
@github-actions
Copy link

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Lock Scope

Verify if locking only old_path in _mv_vector_store_l0_l1 is sufficient to prevent race conditions when moving directory vector indexes, especially if concurrent operations might target new_path.

try:
    async with LockContext(
        get_lock_manager(),
        [old_path],
        lock_mode="mv",
        mv_dst_parent_path=dst_parent,
        src_is_dir=True,
    ):
Query Limit

Confirm that the limit=100 in the filter call for update_uri_mapping will not miss records in cases where a single URI might have more vector index entries (unlikely but possible with multiple levels or other dimensions).

records = await self.filter(filter=And(conds), limit=100, ctx=ctx)

@github-actions
Copy link

PR Code Suggestions ✨

No code suggestions found for the PR.

@qin-ctx qin-ctx merged commit b15b23a into main Mar 19, 2026
5 of 6 checks passed
@qin-ctx qin-ctx deleted the fix/increment_update_dir_vector_store branch March 19, 2026 11:00
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants