Skip to content

Can potentially avoid HEAD call when opening vector indexes #6944

Description

@westonpace

In the open_vector_index_v2 method (lance/src/index/vector.rs) the index files are opened. However, we are not using the known sizes that are present in the manifest. This means we then do a HEAD call to get the size when we are reading those files. This could be avoided if we used open_with_size instead of open:

For example:

let aux_path = index_dir.clone().join(uuid).join(INDEX_AUXILIARY_FILE_NAME);
// The below call should be open_with_size!
let aux_reader = object_store.open(&aux_path).await?;

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions