Skip to content
Discussion options

You must be logged in to vote

The setting you want is maxStopWords, not defaultCompletionOptions.stop.

Continue still builds its own stop list for autocomplete, then trims it before sending the request. In the current codepath the OpenAI-compatible client does body.stop = body.stop?.slice(0, this.getMaxStopWords()), and getMaxStopWords() uses your configured maxStopWords when it is set.

So for a provider that only accepts 5 stop strings, set that on the autocomplete model, for example:

models:
  - name: Cheap autocomplete
    provider: openai
    model: your-model
    roles: [autocomplete]
    apiBase: https://your-provider.example/v1
    maxStopWords: 5

defaultCompletionOptions.stop only changes the actual stop value…

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by MH-Leon
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Help
Labels
None yet
2 participants