-
Notifications
You must be signed in to change notification settings - Fork 167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix quantized example #237
Conversation
Code Metrics Report─────────────────────────────────────────────────────────────────────────────── Language Files Lines Blanks Comments Code Complexity ─────────────────────────────────────────────────────────────────────────────── Rust 70 23339 1550 508 21281 1281 ─────────────────────────────────────────────────────────────────────────────── Total 70 23339 1550 508 21281 1281 ─────────────────────────────────────────────────────────────────────────────── Estimated Cost to Develop 69,864 Estimated Schedule Effort 11.811066 months Estimated People Required 5.038645 ─────────────────────────────────────────────────────────────────────────────── Processed 768517 bytes, 0.769 megabytes (SI) ─────────────────────────────────────────────────────────────────────────────── |
stop_toks: None, | ||
max_len: None, | ||
logits_bias: None, | ||
n_choices: 1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it was defaulting to 0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if this should use the Completion Request best_of
instead. These 2 sources of truth are confusing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lucasavila00 the best_of
parameter sets the number of seqs to consider when they are done by cumulative logprob. It should be <= n_choices
.
stop_toks: None, | ||
max_len: None, | ||
logits_bias: None, | ||
n_choices: 1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lucasavila00 the best_of
parameter sets the number of seqs to consider when they are done by cumulative logprob. It should be <= n_choices
.
Thank you |
I found it while testing #236