|
Hi, They say 'presence_penalty=1.5' will make it better. And what is going to happen if I set 'presence_penalty=50.0' ? Thank you in advance. |
Replies: 1 comment 2 replies
|
The neutral value for
In most implementations, the As a result, the model will almost certainly avoid repeating any token that has already occurred in the prompt or generated output. While this can strongly discourage repetition, it may also lead to unnatural or incoherent text, especially if the model is forced to avoid common or contextually necessary words. |
|
Hi jklj077, btw is there docs related to this kind of parameters and explanations ? if exists, guide me by link. thanks in advance. |
|
The "official" explanations on those parameters can be found at the OpenAI API site: https://platform.openai.com/docs/api-reference/chat/create. AFAIK, the original document containing their technical definition from OpenAI is lost. If you'd like, you could find the implementation of a specific parameter in open-source inference frameworks, e.g., for |
The neutral value for
presence_penaltyis 0.0 and the most API providers limit its range to [-2.0, 2.0].In most implementations, the
presence_penaltyis subtracted from the logits (the raw, unnormalized scores) of any token that has already appeared in the input context. A very high value like 50.0 effectively drives the logits of those tokens far into the negative, making their probabilities negligible after the softmax transformation.As a result, the model will almost certainly avoid repeating any token that has already occurred in the prompt or generated output. While this can strongly discourage repetition, it may also…