You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, each document uploaded requires an API call to get embeddings. Due to API rate-limits, this really slows the document adding process for large corpus.
This process could be parallelized better by implementing a multiple document adding method, which aggregates texts across documents and sends them all to get embeddings in one API call.
The text was updated successfully, but these errors were encountered:
jhohrath
changed the title
Support parallelizing adding docs
Support adding multiple docs at once
Nov 17, 2023
Currently, each document uploaded requires an API call to get embeddings. Due to API rate-limits, this really slows the document adding process for large corpus.
This process could be parallelized better by implementing a multiple document adding method, which aggregates texts across documents and sends them all to get embeddings in one API call.
The text was updated successfully, but these errors were encountered: