Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit of tokens from chat messages? #307

Closed
marvelsanya opened this issue Mar 12, 2024 · 1 comment
Closed

Limit of tokens from chat messages? #307

marvelsanya opened this issue Mar 12, 2024 · 1 comment

Comments

@marvelsanya
Copy link

marvelsanya commented Mar 12, 2024

Every time I try to chat I get an error about exceeding the number of tokens. Example error message:

Error: error in request:{"error":{"message":"This model's maximum context length is 4096 tokens. However, you requested 4833 tokens (4433 in the messages, 400 in the completion). Please reduce the length of either one, or use the "middle-out" transform to compress your prompt automatically.","code":400}}

I tried different characters, including creating my own (which has 1664 tokens in prompts). From the beginning everything is fine, but then after a few short messages there is again an error of exceeding tokens, which means the number of tokens is filled by simple chatting? The more I write in the chat, the more tokens go beyond the limit? How it works? What is needed to solve the problem?

@kwaroran
Copy link
Owner

You edit "Max Context Size" on the settings for that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants