Loading SummaryIndex from files

[pelikhan]

I was trying to modify the save/load example from VectorStoreIndex (https://ts.llamaindex.ai/examples/save_load_index) into using the SummaryIndex (just search/replace). However, it leads to this error.

Error: Cannot initialize SummaryIndex with both nodes and indexStruct

Full repro at https://stackblitz.com/edit/llamaindex-repo1-prbxbm

[marcusschiesser]

@pelikhan Please try again using a different persistDir for the SummaryIndex, e.g.:

 const storageContext = await storageContextFromDefaults({
    persistDir: "./storage_summary",
});
The two different indexes can't share the same directory

[pelikhan]

I've updated the script ( https://stackblitz.com/edit/llamaindex-repo1-xtfjhk?file=index.mjs ). It seems that trying to do a SummaryIndex.fromDocuments on a existing files fails.

[pelikhan]

I noticed that when creating the default storage context from persistDir, it defaults to trying to load the vector_store.json file. However, the summary index does not create this file (or it does not persist it), so the "No valid data found at path: .../vector_store.json starting new store." pops up every time.

vectorStore =

[reidperyam]

@pelikhan I can repro this - were you able to resolve?

[pelikhan]

unfortunately not.

[marcusschiesser]

@pelikhan The problem is that each time you're calling:

const storageContext = await storageContextFromDefaults({
    persistDir: "./storage",
  });
  const index = await SummaryIndex.fromDocuments([document], {
    storageContext,
  });

You're creating a new summary index in the ./storage directory. If you have multiple indexes, loading the index, you'll need to specify which one you want using indexId, e.g.:

const loadedIndex = await SummaryIndex.init({
    storageContext: secondStorageContext,
    indexId: 1,
  });

Otherwise, you will run into the Error: Cannot initialize SummaryIndex with both nodes and indexStruct condition