Doubao-1.5-pro-32k Model Parameter max_tokens should be less than or equal to 12288.0

[zq99299]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I'm using Dify Version 0.15.3. In the workflow orchestration, I selected Doubao-1.5-pro-32k. The same prompt behaves differently between Dify and directly using the Ark SDK (directly calling the official Doubao API):

• When running in Dify, an error occurs: "Model Parameter max_tokens should be less than or equal to 12288.0."
  

• With the official Doubao SDK: It runs normally without any errors. Judging from the API interface response, a total of 10,806 input and output tokens were used.

  

Is this a bug?

✔️ Expected Behavior

It is expected that, similar to the official SDK, there won't be any abnormal errors. I've consulted with the official customer service of Volcengine. They replied that there was no indication of input and output, which can be verified through the SDK access. The prompt used in the SDK was copied from the input in the error screenshot of Dify. The prompts used are exactly the same, yet Dify reports an error.

❌ Actual Behavior

No response

[dosubot]

Hey there, @zq99299! I'm here to help you with any bugs, questions, or contributions you have. Let's get that issue sorted out together.

The issue you're experiencing with the Doubao-1.5-pro-32k model in Dify, where the "max_tokens" parameter should be less than or equal to 12,288, is due to the model configuration in Dify. The configuration sets the max_tokens property to 12,288, which is why you encounter this error when exceeding this limit in Dify. The official Doubao SDK might have a different configuration or no such restriction, allowing the same prompt to work without errors ^[1].

I found similar discussions that might be relevant to your issue:

1. Max Tokens Parameter Not Functioning Correctly - This discussion is still open and might provide additional insights.
2. Max Token Limits Incorrect? - This is another open discussion that could be related to your problem.

These discussions might help you understand if there are any workarounds or additional configurations you can apply.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[zq99299]

Hey there, @zq99299! I'm here to help you with any bugs, questions, or contributions you have. Let's get that issue sorted out together.

The issue you're experiencing with the Doubao-1.5-pro-32k model in Dify, where the "max_tokens" parameter should be less than or equal to 12,288, is due to the model configuration in Dify. The configuration sets the max_tokens property to 12,288, which is why you encounter this error when exceeding this limit in Dify. The official Doubao SDK might have a different configuration or no such restriction, allowing the same prompt to work without errors [1].

I found similar discussions that might be relevant to your issue:

1. Max Tokens Parameter Not Functioning Correctly - This discussion is still open and might provide additional insights.
2. Max Token Limits Incorrect? - This is another open discussion that could be related to your problem.

These discussions might help you understand if there are any workarounds or additional configurations you can apply.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

I've read the two issues mentioned in the text. The questioners might have meant the same problem as mine. However, the responders said they might have confused the concepts. They're right in saying that max_token is a limit on the output tokens, not on the context. But in actual use with Dify, neither my input tokens nor the output tokens exceed 12,288 tokens, yet Dify still reports an error. This is the current problem, and I'm seeking help

Interestingly, when the prompt that caused an error in Doubao 32k/258k is used with ChatGPT - 4o in Dify, it works perfectly fine. So, do they have different limiting logic in Dify? I'm requesting a solution to this problem.

[sinomoe]

Could you please show your model parameter configurations of the LLM node?

[zq99299]

Could you please show your model parameter configurations of the LLM node?

[crazywoola]

• Please open an issue in this repo instead, because we need more information regarding the tool's version.
• https://github.com/langgenius/dify-official-plugins/