-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can´t set indexing_threshold to 0 for bulk upload #620
Comments
Hey @marcossilva It looks that you are using local mode: It doesn't have any index and it not expected to be used with any kind of large loads. Could you please give a try the server version with the same script. |
Thanks for the quick reply @generall . I'm currently using an internally deployed qdrant and had the same problem. I tried with both the memory and path client initializations to debug locally but this also happens either locally (with the in-memory, the path and running a local docker qdrant server) or in the qdrant deployed in our Kubernetes cluster |
Hi @marcossilva , not sure if the issue still persists for you Once you switch from local mode to the server mode, your code should be able to set What is the batch size you're using? |
I'm trying to bulk upload 4.5M points using qdrant but have been strugling with the ingestion time. I tried to run it in memory to speed up, tried to use the upload_collection as suggested in the points docs. But it seems that the main problem is not being able to set the
indexing_threshold
to 0 as sugested in the bulk upload docsCurrent Behavior
On v1.9.0 creating a new collection with the optimizers_config setting the indexing_threshold to 0, updating the collection to set the indexing_threshold to 0 seems to have no effect as the snippet below provide example
Steps to Reproduce
Output:
Expected Behavior
I expected that creating the collection or updating it would set the initial parameters but most of my configurations in the creation of the collection were simply ignored
The text was updated successfully, but these errors were encountered: