-
Notifications
You must be signed in to change notification settings - Fork 294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Slow retrieval of documents #305
Comments
Could you provide the detail? Please do not ignore the issue template. |
When I use my bot, it takes sometimes 17s before starting to render the content. The first step [Retrieve Knowledge] looks ok, but after that, cursor stay at the beginning of the line for a very long time before starting to write the response. This doesn't happen when using the LLM without documents, and it seems to increase with the number of document in the knowledge base. |
Could you check the items in postgres? We can use management console to query and the table definition can be reffered here. |
Could it be related to the instantion of a Lambda function? The behavior only appear on the first inference for a new conversation |
Have you forked and customized this sample? Large container causes longer cold start. You can check whether the invocation is cold or not by reffering cloudwatch logs. |
Ok I have digged a llittle bit on the logs and I found that the POST call to related-documents is very long by itself: Request POST /conversation/related-documents (15:15:32 - 15:15:44) I'm not currently up to date with the codebase and we will perform a full update soon. I'll keep you updated on this. |
@jeremylatorre Thank you for the update. The query part to postgres seems to making the very long latency. For the cost effectiveness, the ACU configured at minimum value by default. Choosing larger value might solve the problem . |
This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 14 days after last commented by someone). And it will be closed automatically 14 days later if not responded. |
This issue has been closed due to no response within 14 days after labeled as "stale", 14 days after last reopened, and 14 days after last commented. |
Describe the bug
When using RAG, I noticed that performances where very low when trying to retrieve information from documents.
Any lead about what is causing this. Clearly the difference between rag and standard bot is huge at that time.
It might also depend on number of documents.
The text was updated successfully, but these errors were encountered: