Repetition penalty passed as frequency penalty in OpenAI-compatible providers

[tommasobonomo]

Bug description

I'm setting the repetition penalty parameter to a set value $p$ on a vLLM-deployed model through chat-ui's .env.local. When inspecting vLLM's logs, I find that the repetition penalty is the default value and instead the frequency penalty is set to $p$, the original value I wanted for the repetition penalty.

I found the cause in the codebase:

chat-ui/src/lib/server/endpoints/openai/endpointOai.ts

Line 169 in d6dc46a

frequency_penalty: parameters?.repetition_penalty,

According to OpenAI's and vLLM's docs, repetition and frequency penalties have a different scale, although they are conceptually similar. The first has a neutral default value of 1.0, whilst the second has a neutral value of 0.0.

This bug introduces a pretty hefty frequency_penalty = 1.0 when the user has no intention of setting a penalty (repetition_penalty = 1.0).

Steps to reproduce

Deploy a model through any OpenAI-compatible endpoint and set the parameter repetition_penalty = 1.0 in the model parameters in .env.local.

Config

MODELS=`[
    {
	"name":"HuggingFaceTB/SmolLM2-1.7B-Instruct",
        "endpoints": [
            {
                "type":"openai",
                "baseURL":"http://vllm_server:8000/v1",
                "completion":"completions"
            }
        ],
        "parameters": {
            "repetition_penalty": 1.0,
        },
    }
]`

Notes

As noted above, the issue stems from this line in the codebase:

chat-ui/src/lib/server/endpoints/openai/endpointOai.ts

Line 169 in d6dc46a

frequency_penalty: parameters?.repetition_penalty,

Repetition and frequency penalty should probably be parsed separately by chat-ui.