[Feature Request] Over 2k token message context support?

[imesha10]

I've noticed that in poe.com that if I send a very large text over 2000 tokens I think it was, the poe bot does not respond and does not respond in the website.

My question and feature request is does this project support over 2000k context on gpt4 (I think I already tried and it failed on this repo), if not would it be possible to support it?

[snowby666]

It depends on the text to be honest. Sometimes the model can understand the context given and generate a response that makes sense, even if the text is over 2000 tokens. However, other times the model may struggle to understand the context, especially if the text is very long or contains complex language. In these cases, it's possible that the model will not be able to generate a response that accurately addresses the prompt.

[snowby666]

And the token limit on poe.com is just relatively correct since the devs added up a pre-prompt engineering process before sending the request.