create_index silently ignores num_partitions when ivf_centroids_file is also given. The file's cluster count wins, with no error and no warning:
ds.create_index(
["vector"], index_type="IVF_PQ",
ivf_centroids_file="centroids.npy", # 4 clusters
num_partitions=8,
num_sub_vectors=8,
)builds a 4-partition index.
The branch loads the file and then overwrites the caller's value (python/python/lance/dataset.py:4048):
ivf_centroids = np.load(f)
num_partitions = ivf_centroids.shape[0]
The only mismatch guard in this function is on the numpy-array path (:4108-4111), and by the time it runs num_partitions has already been replaced by the file's count, so it compares the value against itself and can never fire for the file path.
Nothing documents which one is supposed to win. num_partitions has a docstring entry that marks it deprecated in favor of target_partition_size (:4268); ivf_centroids_file has no docstring entry anywhere. So a caller who passes both has no way to know that one of the two arguments was discarded.
create_indexsilently ignoresnum_partitionswhenivf_centroids_fileis also given. The file's cluster count wins, with no error and no warning:builds a 4-partition index.
The branch loads the file and then overwrites the caller's value (python/python/lance/dataset.py:4048):
The only mismatch guard in this function is on the numpy-array path (:4108-4111), and by the time it runs
num_partitionshas already been replaced by the file's count, so it compares the value against itself and can never fire for the file path.Nothing documents which one is supposed to win.
num_partitionshas a docstring entry that marks it deprecated in favor oftarget_partition_size(:4268);ivf_centroids_filehas no docstring entry anywhere. So a caller who passes both has no way to know that one of the two arguments was discarded.