Weird default API parameters for Anthropic models

[hourianto]

What happened?

LibreChat has very weird default parameters for Anthropic models, with them being:

• Temperature at 1
• Top P at 0.7
• Top K at 5

From what I know, most uses of Claude models should by default only change the temperature and not touch other values, so that most requests are done with temperature of e.g. 0.4 and 0.5, and top_p and top_k aren't changed to let the API use the default values.

And, to quote Anthropic themselves from https://docs.anthropic.com/en/api/messages:

temperature - Amount of randomness injected into the response. Defaults to 1.0. Ranges from 0.0 to 1.0. Use temperature closer to 0.0 for analytical / multiple choice, and closer to 1.0 for creative and generative tasks. Note that even with temperature of 0.0, the results will not be fully deterministic.

top_k - ... Recommended for advanced use cases only. You usually only need to use temperature.

top_p - ... You should either alter temperature or top_p, but not both.

So yeah, I'm really curious as to why this is done, when in most cases applications just set the temperature to a value closer to 0 like 0.4 and don't touch those other parameters? And, if this is done deliberately, can you please add a way to NOT set those values? The Anthropic docs do not specify the default values for top_p and top_k, so I think it should be possible to just set the temperature and let the API take care about other parameters in case I don't agree with the defaults that LibreChat sets.

Steps to Reproduce

Select Anthropic, go into preset settings, observe the weird values.

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Moved to suggestions. These “weird” values are just the old defaults. If they remain the same (untouched) then they get removed from the API payload anyway. These need to be updated to the latest defaults according to the API docs you linked

[hourianto]

Oh, okay, thanks for the clarification, it's just that I don't remember Claude models ever having such defaults. As for the latest defaults - by default I think LibreChat should just set temp to something like 0.5 maybe? And other parameters should be separately configurable, so it's not confusing if they're active or not (as it is currently)

[jmckernan95]

Also confused about this! So if I leave Top P and Top K at default, but modify the temperature, add custom instructions etc, will Top P and Top K not get added to the API payload? Or does setting a Preset mean they will be altered?

[rahuldave]

Whats the story with this right now? Leave at default for top p and top k and they are not used?

[danny-avila]

If left at default, they are not included. We assume those values to be the default determined by the provider. If they are updated, whatever you set them to outside of those values will be provided in the payload.