perf(fts): search partitions in pipelined chunks#7950
Conversation
📝 WalkthroughWalkthroughBM25 inverted-index searches now process partitions in configurable chunks. Postings load concurrently within each chunk, while scoring runs sequentially in one CPU task and omits partitions without postings hits. ChangesChunked FTS search
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant InvertedIndex
participant AsyncLoader
participant CpuScorer
InvertedIndex->>AsyncLoader: Load postings for each partition in a chunk
AsyncLoader-->>InvertedIndex: Return non-empty partition data
InvertedIndex->>CpuScorer: Score loaded partitions sequentially
CpuScorer-->>InvertedIndex: Return non-empty PartitionCandidates
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
One tiny cpu-pool task per partition (~350 per query) interleaves so heavily under concurrent load that scoring loses cache locality and the dispatch rate floods the blocking pool. Partitions are now searched in chunks of LANCE_FTS_SEARCH_CHUNK (default 16): each chunk loads its postings and scoring DocSets concurrently, then a single cpu task scores the chunk's partitions back-to-back on one thread; chunks pipeline against each other through buffer_unordered. On a 320-core node against a 100M-doc 42-language corpus (5-term OR, k=100, warm), per-query scoring CPU drops ~5x and c16 throughput doubles (227 -> 428 qps at half the latency); combined with a contention-free cache read path the chunked layout reaches 1340 qps at c128 versus 345 for per-partition tasks.
01ca66d to
34e98d5
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@rust/lance-index/src/scalar/inverted/index.rs`:
- Around line 1000-1004: Replace the unbounded try_join_all fan-out in the chunk
loading path with a bounded unordered stream using self.store.io_parallelism()
as the concurrency limit, while preserving error propagation and flattening all
loaded posting lists into the existing loaded collection.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: QUIET
Plan: Pro Plus
Run ID: 60bc4dba-548e-4233-bf90-c85d39ea88f3
📒 Files selected for processing (1)
rust/lance-index/src/scalar/inverted/index.rs
|
Good catch on the arithmetic — the chunked layout does raise the number of pending load futures. We benchmarked the concern directly and added a bound; data below (320-core node, 100M-doc 42-language index, ~350 partitions/query, contention-free cache backend so cache effects don't mask load effects). 1. Cold path (empty cache, fresh process) — chunking helps, doesn't flood:
The reason pending futures don't translate into storage pressure: actual in-flight IO is bounded by the store's scheduler ( 2. Chunk-concurrency bound sweep (warm, the bound you suggested):
The 32-vs-unbounded row is identical by construction, so its 11% delta bounds run-to-run noise at ~±6% — every difference in the table is inside that band. The bound is effectively free, so the code now uses Chunk size itself was swept too (4/8/16/32 at c16 and c128): 16 is the knee (4 costs 24% of peak throughput, 32 is flat vs 16), hence the default. |
a5678ca to
84d8fe5
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rust/lance-index/src/scalar/inverted/index.rs (1)
105-113: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winReject invalid
LANCE_FTS_SEARCH_CHUNKvalues instead of silently defaulting.Malformed values and
0are silently replaced with16, hiding configuration mistakes and making the effective concurrency differ from the requested value. Keep the default only for an unset variable; reject present invalid values withError::invalid_inputthat includes the variable name and raw value, then propagate the error frombm25_search.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/lance-index/src/scalar/inverted/index.rs` around lines 105 - 113, Update fts_search_chunk to return a Result and preserve 16 only when LANCE_FTS_SEARCH_CHUNK is unset; for malformed or zero values, return Error::invalid_input including the variable name and raw value. Propagate this Result through bm25_search and update its callers as needed.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@rust/lance-index/src/scalar/inverted/index.rs`:
- Around line 105-113: Update fts_search_chunk to return a Result and preserve
16 only when LANCE_FTS_SEARCH_CHUNK is unset; for malformed or zero values,
return Error::invalid_input including the variable name and raw value. Propagate
this Result through bm25_search and update its callers as needed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: QUIET
Plan: Pro Plus
Run ID: 8147fe44-0f0f-47c8-9793-6fee2b491e63
📒 Files selected for processing (1)
rust/lance-index/src/scalar/inverted/index.rs
1 participant
Problem
InvertedIndex::bm25_searchspawns one cpu-pool task per partition (~350 per query on a 100-shard index). Under concurrent load this has two costs:index_comparisons(ANALYZE-verified — the comparison count is unchanged; the cost per comparison is what inflates).Change
Partitions are searched in chunks of
LANCE_FTS_SEARCH_CHUNK(default 16):buffer_unordered, so chunk N scores while chunk N+1 loads.A secondary benefit: partitions scored consecutively on one thread observe each other's published shared top-k floor immediately.
LANCE_FTS_SEARCH_CHUNK=1restores the old one-task-per-partition behavior.Results
320-core node, 100M-doc 42-language corpus, 5-term
match_any(OR), tier-100-200 words, k=100, warm + prewarmed, exact scoring, 180s windows, bench returns_rowid+_scoreonly:At c64 throughput is capped by a separate cache-read bottleneck (a follow-up PR swaps the backend); the chunked layout cuts the CPU burned per query ~5x (1.11 → 0.20 core-seconds), which that follow-up then converts into throughput: with the contention-free read path the chunked layout reaches 1340 qps at c128 vs 345 for per-partition tasks.
Tests
cargo test -p lance-index scalar::inverted— 332 passed. The multi-partition tests (41 partitions = 3 chunks) exercise the multi-chunk merge path with exact row_id assertions.