How to index collection using generator function?

[shubham526]

I have a large collection of 16 million passages that I want to index. It's not practical to keep all documents and ids in memory as a list to pass it to the index function. Is there a way to index large collections using generators?