-
Notifications
You must be signed in to change notification settings - Fork 270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
prompts get too large #16
Comments
I closed issue #49 since this issue is more specific, we can use this issue to discuss how to further implement summarization 🙌. About your comment in PR #52: I can test other chain_type to see if I get the same problem with large files. I'm going to dig a little deeper into the docs that you left (about llama-index) to understand better how to implement the custom summary chain, but if I understand correctly the basic idea is:
|
@calebgcc I introduced in the rabbit_hole a TextSplitter that can be customized (chunk_size and chunk_overlap). So Cat users can decide themselves how long they want their text chunks. You find in |
@pieroit I was trying to trigger this error, but I think summarizing and chunking the documents solved it. The documents that are retrieved from the cat are often too small to cause problems, and this adds up to the fact that k is by default 5. Maybe prompt summarization is no longer necessary, let me know how to proceed, for example we can try increasing the value of k to see how it affects the prompt. |
Increasing k is a good test, also if somebody uploads a doc and chooses a large chunk size the problem remains. There should be a check before inserting memories in the prompt, if they are "too long" they should be summarized. We can postpone the problem and close this issue as we are mostly covered, or if you feel like it also tackle the above. Thanks 🙏 |
Use more deeply langchain routines to keep the prompt at limited length (CombineDcoumentsChain etc.).
Summarization may also be appropriated when documents are uploaded.
The text was updated successfully, but these errors were encountered: