-
Notifications
You must be signed in to change notification settings - Fork 26.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pipeline of "text-generation" with model "meta-llama/Llama-2-7b-chat-hf" doesn't respect temperature #25326
Comments
|
Hi @kechan, thanks for raising this issue! You should pass |
@amyeroberts Thanks for the help. I totally missed this one! wonder why this is needed if this is known from values of T, top_k, or top_p. |
Best short explanation is this comment: #22405 (comment) Generate is a very powerful functionality that's had lots of arguments and logic added over time. @gante's doing a lot of work - refactoring, docs, demos - to make this easier for people to use, but there's always a balance between simplifying and keeping backwards compatibility of behaviours :) |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
System Info
transformers
version: 4.28.1Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
temperature=0.9,
top_k=50,
top_p=0.9,
max_length=500)
Expected behavior
Expect random variation in generated text with each run.
The text was updated successfully, but these errors were encountered: