Skip to content

fix(index): preserve fts prewarm position codec#6485

Merged
BubbleCal merged 1 commit intomainfrom
yang/fix-fts-prewarm-codec
Apr 11, 2026
Merged

fix(index): preserve fts prewarm position codec#6485
BubbleCal merged 1 commit intomainfrom
yang/fix-fts-prewarm-codec

Conversation

@BubbleCal
Copy link
Copy Markdown
Contributor

Bug Fix

What is the bug?

FTS v2 indices built with with_position=true can reconstruct cached posting lists with the wrong shared-position codec during prewarm. The prewarm path rebuilds PostingList values from projected RecordBatch objects and re-infers the positions codec from batch metadata, even though the reader has already resolved the correct partition-level positions_layout.

What issues or incorrect behavior does the bug cause?

After prewarm_index(..., with_position=True), phrase queries can fail even though the same query succeeds before prewarm. In practice this shows up as shared position stream decode failures in the cached path because PackedDelta data can be interpreted as the legacy codec.

How does this PR fix the problem?

This PR threads positions_layout from PostingListReader through the prewarm reconstruction path into CompressedPostingList::from_batch, so cached postings reuse the already-parsed shared-position codec instead of guessing from projected batch metadata. It also adds a regression test that covers the V2 + positions + tail-remainder case and verifies prewarm preserves correct phrase-query behavior.

Validation

  • cargo test -p lance-index test_prewarm_with_ -- --nocapture

@github-actions github-actions Bot added the bug Something isn't working label Apr 11, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 11, 2026

Codecov Report

❌ Patch coverage is 94.06780% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance-index/src/scalar/inverted/index.rs 94.06% 6 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@BubbleCal BubbleCal marked this pull request as ready for review April 11, 2026 09:21
@BubbleCal BubbleCal merged commit 0634c6a into main Apr 11, 2026
30 checks passed
@BubbleCal BubbleCal deleted the yang/fix-fts-prewarm-codec branch April 11, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants