No embeddings found in folder

[loretoparisi]

I'm using autofaiss==2.15.8. I get the error No embeddings found in folder when calling build_index:

from autofaiss import build_index

 max_index_memory_usage='4G'
        build_index(embeddings=embeddings,
                    index_path='%s/knn.index'%path_to_store_index,
                    index_infos_path="%s/knn_info.json"%path_to_store_index,
                    max_index_memory_usage=max_index_memory_usage)

Here I use the embeddings parameters that according to the docs may contain a path or the object (numpy array), and in the code above I'm using the latter approach, that according to the source of build_index here

if isinstance(embeddings, np.ndarray):
        tmp_dir_embeddings = tempfile.TemporaryDirectory()  # pylint: disable=consider-using-with
        np.save(os.path.join(tmp_dir_embeddings.name, "emb.npy"), embeddings)
        embeddings_path = tmp_dir_embeddings.name
    else:
        embeddings_path = embeddings  # type: ignore

should creare a file within the tmp_dir_embeddings , that afterwards causes the error

No embeddings found in folder /tmp/tmpnt82l_ms

there I assume the tmp_dir_embeddings is the one specified in the error, /tmp/tmpnt82l_ms, while the instance type is of np.ndarray.

[loretoparisi]

[UPDATE]

Digging into dependant libraries source code, I have found that the error is raied by the embedding-reader module here in the NumpyReader class:

class `NumpyReader`:
    """Numpy reader class, implements init to read the files headers and call to procuce embeddings batches"""

    def __init__(self, embeddings_folder):
        self.embeddings_folder = embeddings_folder
        self.fs, embeddings_file_paths = get_file_list(embeddings_folder, "npy")

        headers = get_numpy_headers(embeddings_file_paths, self.fs)
        self.headers = pd.DataFrame(
            headers,
            columns=["filename", "count", "count_before", "dimension", "dtype", "header_offset", "byte_per_item"],
        )

        self.count = self.headers["count"].sum()
        if self.count == 0:
            raise ValueError(f"No embeddings found in folder {embeddings_folder}") # <--- this error

[loretoparisi]

Fixed downgrading embedding_reader to back support Python 3.7.x see - rom1504/embedding-reader#44