Multi topic Best practice

[DanSan23]

I'm working on building an index to efficiently query documents. With a single document, everything seems to work fairly well, but as the number of documents increases, the performance degrades. Can you tell me the best practices to deal with this scenario? The biggest problems occur with ambiguous queries where the answer could reside in several different documents.

[Disiok]

I think this might be a relevant example to take some inspiration from: https://github.com/jerryjliu/llama_index/blob/main/examples/composable_indices/city_analysis/City_Analysis-Unified-Query.ipynb

tldr: you can define sub-index for each document (or collection of documents), then define a (LLM powered) router to automatically choose between the different sub-indices.

[dosubot]

Hi, @DanSan23! I'm helping the LlamaIndex team manage their backlog and wanted to let you know that we are marking this issue as stale.

From what I understand, you were seeking advice on improving the performance of your document query index as the number of documents increases. Disiok suggested taking inspiration from an example in the llama_index repository, where sub-indices are defined for each document or collection of documents, and a router automatically chooses between them. It looks like you reacted positively to this suggestion with a thumbs up.

Before we close this issue, we wanted to check if it is still relevant to the latest version of the LlamaIndex repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your contribution to the LlamaIndex project!