Skip to content

bug: row_range panics where partition_size returns 0 for an absent partition #9012

Description

@LuciferYang

IvfModel::row_range and IvfModel::partition_size answer the same question about the same input and disagree: for a partition the model does not carry, one panics and the other returns 0.

pub fn row_range(&self, partition: usize) -> Range<usize> {
    let start = self.offsets[partition];          // storage.rs:144, panics
    let end = start + self.lengths[partition] as usize;
    start..end
}

pub fn partition_size(&self, part: usize) -> usize {
    self.lengths.get(part).copied().unwrap_or_default() as usize   // storage.rs:94, returns 0
}

num_partitions() adds to the confusion: it prefers centroids.len() and only falls back to offsets.len() (storage.rs:86-91), so the count a caller iterates over is not necessarily the one row_range can index.

I could not construct the panic through the public API, and this is worth saying plainly rather than filing it as a live bug. User-supplied partition ids reach load_partition, which rejects out-of-range ids with a clean error, and search-path ids come from find_partitions over the centroids so they are below nlist. The three callers that index without a guard would need a written model whose num_partitions() exceeds offsets.len(), and neither the builder nor the distributed merger can emit one.

The one seam I could not rule out: nothing checks the centroid count against lengths.len() on write or on load, and rust/lance/src/index/vector/ivf.rs:2690 grafts centroids from a different file than the one supplying the lengths, so a mismatched shard set there would reach the panic. I could not produce that from a normal write path.

So this is a contract inconsistency between two neighbouring methods rather than a reachable failure.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions