-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix for silent llama #279
Fix for silent llama #279
Conversation
Fix for prioritization agent deleting priority list when returning empty output. Prompt engineering and parameter improvements to reduce number of empty responses from llama models. More debug output.
Attached is a rather lengthy example of the sort of output I get with this PR. More than anything, it illustrates the difficulties that my local model has staying on task and how these issues get propagated and/or handled internally with subsequent iteration. The propensity to produce empty output from time to time may be a context length issue, and I believe that further optimization in this regard is possible. Nonetheless, I feel that output has gotten much better. As ever, comments, criticisms and suggestions are greatly appreciated! |
This fixed my mlock issue. Current;y running for first time. |
@dibrale Thanks, that solves the issue for me. |
@jmtatsch - To answer your questions: 1. I hope this clarifies my reasoning a bit. Please let me know if you have any other questions or suggestions. |
Can we change the CTX_MAX and max_tokens only for Llama? Also, there are bunch of changes to the prompts, which I'd like to split into a separate PR, if possible, as I am also about to merge another prompt refactor/change from BabyBeeAGI. |
CTX_MAX only gets set for Llama models anyway, unless I am mistaken. Its assignment occurs behind a branch that checks for llama, and this behavior does not change in my PR. |
At the request of the maintainer
@francip - I've reverted the prompt changes, so hopefully this PR meets your needs. I'll submit prompt changes for separate consideration. |
Prompt changes are now in a separate PR as requested. Please let me know if there are any further issues. |
Fix for prioritization agent deleting priority list when returning empty output. Prompt engineering and parameter improvements to reduce number of empty responses from llama models. More debug output.
@jmtatsch - This should fix the issue you experienced with my prior PR.