Set min/max values for float options for OpenAI models

[simonw]

No description provided.

[simonw]

I tried this in a branch:

llm/llm/default_plugins/openai_models.py

Line 91 in 6bad6de

temperature: Optional[confloat(ge=0, le=2)] = Field(

It has an unfortunate impact on the output of llm models list --options:

llm/docs/usage.md

Lines 102 to 118 in 6bad6de

	OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
	temperature: Annotated[float, None, Interval(gt=None, ge=0, lt=None, le=2), None, None]
	What sampling temperature to use, between 0 and 2. Higher values like
	0.8 will make the output more random, while lower values like 0.2 will
	make it more focused and deterministic.
	max_tokens: int
	Maximum number of tokens to generate.
	top_p: Annotated[float, None, Interval(gt=None, ge=0, lt=None, le=1), None, None]
	An alternative to sampling with temperature, called nucleus sampling,
	where the model considers the results of the tokens with top_p
	probability mass. So 0.1 means only the tokens comprising the top 10%
	probability mass are considered. Recommended to use top_p or
	temperature but not both.
	frequency_penalty: Annotated[float, None, Interval(gt=None, ge=-2, lt=None, le=2), None, None]
	Number between -2.0 and 2.0. Positive values penalize new tokens based
	on their existing frequency in the text so far, decreasing the model's
	likelihood to repeat the same line verbatim.

Note the ugly:

temperature: Annotated[float, None, Interval(gt=None, ge=0, lt=None, le=2), None, None]

I'm not sure how best to clean that up, especially since that output differs between Python versions.

In fact https://github.com/simonw/llm/actions/runs/5583462965/jobs/10203805843 failed on Python 3.7 with this differing cog output:

  temperature: typing_extensions.Annotated[float, None, Interval(gt=None, ge=0, lt=None, le=2), None, None]

[simonw]

In the meantime it works OK without those min/max settings - the prompt ends up being sent to the OpenAI API which returns a useful error message:

llm -m chatgpt 'say hello twice' -o temperature -1

Error: -1.0 is less than the minimum of 0 - 'temperature'

[pavelkraleu]

What about something like this @simonw ?