Wrong max tokens limit for OpenRouter Sonnet 3.5

[infinitnet]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.9.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Connect OpenRouter and select Sonnet 3.5 in an LLM node.

✔️ Expected Behavior

Max tokens limit should be 8192 (see https://openrouter.ai/models/anthropic/claude-3.5-sonnet)

❌ Actual Behavior

Max tokens limit is 4096 artificially limiting the output length to 50% of what's actually possible when Sonnet 3.5 is used through OpenRouter.

[dosubot]

I found a similar discussion regarding incorrect token limits: Max Token Limits Incorrect?. This discussion is currently unsolved and was last updated on August 14, 2024 ^[1].

To address the issue you're facing, you can update the token limit in the Dify configuration file for OpenRouter Sonnet 3.5. Modify the max_tokens parameter in the api/core/model_runtime/model_providers/openrouter/llm/claude-3-5-sonnet.yaml file. Change the default and max values to 8192 to match the expected limit ^[2].

To continue talking to Dosu, mention @dosu.