Summary
VectorStore.from_filespace() fails when loading a vector store whose metadata.json has "batch_size": null (as produced by v1.0.0, which did not persist batch_size). This happens when the batch_size argument is None which is the default argument.
Steps to Reproduce
Using the DEMO/general_workflow_demo.ipynb as a start we can add the following after creating my_vector_store:
# 1. Create a VectorStore and manually simulate a v1.0.0 metadata file (or any file missing the batch_size)
my_vector_store = VectorStore(
file_name="data/testdata.csv",
data_type="csv",
vectoriser=vectoriser,
output_dir="testdata",
overwrite=True,
)
import json
with open("testdata/metadata.json", "r") as f:
metadata = json.load(f)
metadata["batch_size"] = None # simulates v1.0.0 metadata
with open("testdata/metadata.json", "w") as f:
json.dump(metadata, f, indent=2)
# 2. Attempt to reload - raises IndexBuildError
VectorStore.from_filespace(folder_path="testdata", vectoriser=vectoriser)
Expected Behaviour
from_filespace() loads successfully, falling back to a sensible default batch_size when the metadata value is null.
Actual Behaviour
IndexBuildError: Failed to initialise VectorStore instance from filespace.
context={"cause_message": "type object 'VectorStore' has no attribute 'batch_size'", "cause_type": "AttributeError", ...}
Root Cause
In src/classifai/indexers/main.py, the from_filespace(), batch_size is assigned as:
vector_store.batch_size = batch_size if batch_size is not None else metadata["batch_size"]
When both batch_size (the argument) and metadata["batch_size"] are None, vector_store.batch_size is set to None. Any subsequent call to .search() then fails because batch_size is used as a range step, which does not accept None.
Fix
from_filespace() should fall back to the same default as __init__ (128) when both values are None. The default should be defined in a single place to avoid duplication.
Affected Versions
- Introduced in: v1.1.0
- Affects: any store created with v1.0.0 loaded by v1.1.0+
Summary
VectorStore.from_filespace()fails when loading a vector store whosemetadata.jsonhas"batch_size": null(as produced by v1.0.0, which did not persistbatch_size). This happens when thebatch_sizeargument isNonewhich is the default argument.Steps to Reproduce
Using the
DEMO/general_workflow_demo.ipynbas a start we can add the following after creatingmy_vector_store:Expected Behaviour
from_filespace()loads successfully, falling back to a sensible defaultbatch_sizewhen the metadata value isnull.Actual Behaviour
Root Cause
In
src/classifai/indexers/main.py, thefrom_filespace(),batch_sizeis assigned as:When both
batch_size(the argument) andmetadata["batch_size"]areNone,vector_store.batch_sizeis set toNone. Any subsequent call to.search()then fails becausebatch_sizeis used as a range step, which does not acceptNone.Fix
from_filespace()should fall back to the same default as__init__(128) when both values areNone. The default should be defined in a single place to avoid duplication.Affected Versions