-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When the text gets longer, the embedding API does not seem to work. #19
Comments
Are you using the default OpenAI embeddings? Is there an error message or is the code stalling? |
I have noticed that you have already added retry decorators, but the 429 response is still being triggered. Only some open-source embedding methods seem to work. @Retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6)) If I shortened the text, the embedding API would work. However, if the text gets longer, the following information must be shown: 2024-03-15 07:46:21,911 - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests" |
It seems like you're running into rate limiting issues due to OpenAI's API. You could request a rate limit increase from OpenAI if they allow it. Also, currently, we utilize multithreading to build the leaf nodes. You can switch off multithreading, which will make it slower but should help avoid hitting the rate limits. To make this change, update the following line in raptor/raptor/RetrievalAugmentation.py Line 219 in 2e3e83e
From: self.tree = self.tree_builder.build_from_text(text=docs) To: self.tree = self.tree_builder.build_from_text(text=docs, use_multithreading=False) |
Strange! It works fine when I run the demo for the first time. But when I rerun the demo, an error occurred, the text in demo seems too long for raptor. EmbedModel, QAModel and SummModel are all custom. Follow this, the question are solved. Update: |
No description provided.
The text was updated successfully, but these errors were encountered: