-
Notifications
You must be signed in to change notification settings - Fork 186
Open
Description
I have converted my table data into Document format. Page content with custom metadata, here the metadata is quite large. The issue am facing is i was not able load the data into qdrant db sometimes, it is raising "Timeout" error. But the main issue is this is inconsistent, sometimes it loads successfully when i load 100 Documents in one go(using langchain), sometimes its loding only when i load 10 documents at a time, in those scenarios i have written custom batch wise upload code. But how can I know which number of documents can be loaded. I want to know how can i solve this and why is this happening???
- I tried increasing timeout and setting https as true, but it didn't solve the issue
- Is this is a bug or something?
Below is the error am getting
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "qdrant_db_main.py", line 153, in load_to_vectordb
raise e
File "qdrant_db_main.py", line 146, in load_to_vectordb
batch_ids = qdrant_instance.add_documents(documents=batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/langchain_core/vectorstores/base.py", line 286, in add_documents
return self.add_texts(texts, metadatas, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/langchain_qdrant/qdrant.py", line 444, in add_texts
self.client.upsert(
File "venv/lib/python3.12/site-packages/qdrant_client/qdrant_client.py", line 1567, in upsert
return self._client.upsert(
^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/qdrant_remote.py", line 1908, in upsert
http_result = self.openapi_client.points_api.upsert_points(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api/points_api.py", line 987, in upsert_points
return self._build_for_upsert_points(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api/points_api.py", line 512, in _build_for_upsert_points
return self.api_client.request(
^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api_client.py", line 89, in request
return self.send(request, type_)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api_client.py", line 106, in send
response = self.middleware(request, self.send_inner)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api_client.py", line 215, in __call__
return call_next(request)
^^^^^^^^^^^^^^^^^^
File "venv/lib/python3.12/site-packages/qdrant_client/http/api_client.py", line 118, in send_inner
raise ResponseHandlingException(e)
qdrant_client.http.exceptions.ResponseHandlingException: timed out`
Metadata
Metadata
Assignees
Labels
No labels