Keep hitting openai rate limit

[SingL3]

Hi, currently the memgpt is the only app doing requests to openai and it keep raising this and Thinking:

acreate (backoff): caught error: Rate limit reached for gpt-4 in organization org-uL6ErIyeknQXybIzTsylefqn on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us
through our help center at help.openai.com if you continue to have issues.

Is there any limit for the doc or something? The length of the doc or whatever?

[SingL3]

After 20 retries, I got this:

openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 23284 tokens (22565 in the messages, 719 in the functions). Please reduce the length of the messages or functions.

So maybe I need to shorten my doc？

[tractorjuice]

Check your OpenAI rate limits at platform.openai.com

[SingL3]

@tractorjuice I am the only user of my openai account, so I am confused about this. That means the Memgpt itself hit the rate limit.

[tractorjuice]

@tractorjuice I am the only user of my openai account, so I am confused about this. That means the Memgpt itself hit the rate limit.

Or you have a very small quota for API calls. Check your limits.

Due to the MemGPT architecture, it is hungry for API calls and tokens 😁.

[tractorjuice]

@tractorjuice I am the only user of my openai account, so I am confused about this. That means the Memgpt itself hit the rate limit.

Or you have a very small quota for API calls. Check your limits.

Due to the MemGPT architecture, it is hungry for API calls and tokens 😁.

Sign up to PromptLayer and push the API calls through there. You will see every single call in detail + latency.

[cpacker]

Are you completely sure you're the only one using that OpenAI API org?

If you go to https://platform.openai.com/account/usage, you can click down into "Daily Usage" and see the exact requests made and their timestamps.

MemGPT will launch requests of up to ~8k tokens (the first request with an "empty" agent is ~3k) - I think most people have a 10k/min token limit on gpt-4, so this means that technically MemGPT can run only ~1-3 times per minute. However in practice, we've found that if you're the only person using an OpenAI API account with GPT-4 access, you shouldn't hit the rate limit that often (in fact MemGPT should feel quite smooth with minimal extended latency due to backoff on rate-limited requests).

If this problem is still affecting you and you've determined that (1) you have a token/minute limit of 10k for gpt-4, (2) you are 100% the only person using the account as shown in the daily breakdown, please reopen the issue (and maybe DM us a screenshot of the usage on Discord, or post a full output of the MemGPT command you ran here on GitHub) and we will take a closer look!

[jms90h5]

After updating my install of pymemgpt and regenerating the archive I'm still getting rate limit errors. Note that the queries do eventually succeed, although sometimes the generated messaged exceed the context limit. Please see the attached screenshot. This is the only session accessing the OpenAI API for my account.