"max context token" not work for anthropic models.

[2p990i9hpral]

What happened?

"max context token" function with the anthropic model does not work.
the "max context token" works well with the openAI model (gpt-4o). However, If I set the value for the anthropic model (sonnet-3.5) It doesn't work regardless of the size (I tried 2000 and 4000) but only works with the default option (System.)

Steps to Reproduce

1. open new chat with anthropic agent (e.g. sonnet-3.5) and "max context token" option.
2. send a message
3. error occurs.

What browsers are you seeing the problem on?

No response

Relevant log output

Something went wrong. Here's the specific error message we encountered: Operation failed after 3 attempts: 400 {"type":"error","error":{"type":"invalid_request_error","message":"messages: at least one message is required"}}
This is the setting.

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

If you're setting context this low, you need to match the output tokens respectively. It's recommended you use a preset for this as well to avoid setting it every time.

From your screenshot, you have output tokens set to 8192 and the context gets screwed up to -6192.

if you want to give the model 1500 response tokens, your settings should be:

Max Context Tokens: 2000
Max Output Tokens: 1500

this by proxy limits the user prompt, with or without any previous messages, to 500 tokens.

[2p990i9hpral]

Thank you for your quick response.
It seems there's a difference in setting between endpoints. I'll try it later.

[danny-avila]

yup, 3.5-sonnet has a max of 8192 response tokens, and OpenAI's "default" is 1024 (not sure what the max is).

Right now, I may need to double check if only way to use that 8192 total response is if we explicitly set the value, because not setting it for you could prevent the issue (the lengthy response capability is in Beta per Anthropic API).

[2p990i9hpral]

yup, 3.5-sonnet has a max of 8192 response tokens, and OpenAI's "default" is 1024 (not sure what the max is).

Right now, I may need to double check if only way to use that 8192 total response is if we explicitly set the value, because not setting it for you could prevent the issue (the lengthy response capability is in Beta per Anthropic API).

I'm still not sure about the procedure.

If I set the parameters as:
Max Context Tokens: 2000,
Max Output Tokens: 1500
then the model's maximum context is 500 and the response is 1500, am I correctly understanding?

For example, If I set Max Context Tokens: 1000, and Max Output Tokens: 500 and use a 400 tokens-length prompt, Does it work with only the prompt without context or does It just forget the prompt after a few chats?

[danny-avila]

It will just forget the prompt after a few chats if your prompt (the current input and any previous messages and custom instructions) cannot fit in 500 token-length remainder for the prompt.