Rate limit problem

[localjo]

I'm trying to follow the example here, and getting an error. To get the example to work, I had to add my API key, so I just did openai.api_key = "sk-mykey, but now I am getting the following error:

RateLimitError                            Traceback (most recent call last)
File ~/opt/anaconda3/lib/python3.9/site-packages/tenacity/__init__.py:407, in Retrying.__call__(self, fn, *args, **kwargs)
    406 try:
--> 407     result = fn(*args, **kwargs)
    408 except BaseException:  # noqa: B902

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/embeddings_utils.py:23, in get_embedding(text, engine)
     21 text = text.replace("\n", " ")
---> 23 return openai.Embedding.create(input=[text], engine=engine)["data"][0]["embedding"]

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/api_resources/embedding.py:34, in Embedding.create(cls, *args, **kwargs)
     33 try:
---> 34     response = super().create(*args, **kwargs)
     36     # If a user specifies base64, we'll just return the encoded string.
     37     # This is only for the default case.

File ~/opt/anaconda3/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py:115, in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
    114 url = cls.class_url(engine, api_type, api_version)
--> 115 response, _, api_key = requestor.request(
    116     "post",
    117     url,
    118     params=params,
    119     headers=headers,
    120     stream=stream,
...
--> 361     raise retry_exc from fut.exception()
    363 if self.wait:
    364     sleep = self.wait(retry_state=retry_state)

RetryError: RetryError[<Future at 0x179356970 state=finished raised RateLimitError>]

I have credit in my account, and only ~300 short sentences in my CSV. I'm surprised to be running into a rate limit, but is there a recommended way to throttle requests for this example? I should add that I'm not a Python developer, I'm just trying to figure out how to replace the search API with the embeddings API, and since the only documentation for that replacement is a Python example, I'm just trying to get the example to run so I know what I need to do in my own app.

[ted-at-openai]

Thanks for flagging this. The get_embedding function has some retry/back-off behavior built-in, so I'm surprised you hit the rate limit. Maybe we need to make the back-off a little more conservative - I'll look into this. In the meantime, one thing you can do is to redefine get_embeddings() to have a longer delay between retries. Here's the code from the OpenAI Python library, which uses the tenacity library for retries:

from tenacity import retry, stop_after_attempt, wait_random_exponential

import openai


@retry(wait=wait_random_exponential(min=1, max=20), stop=stop_after_attempt(6))
def get_embedding(text: str, engine="text-similarity-davinci-001") -> List[float]:

    # replace newlines, which can negatively affect performance.
    text = text.replace("\n", " ")

    return openai.Embedding.create(input=[text], engine=engine)["data"][0]["embedding"]

You can copy and paste this code, set max to a higher number like 60, and use your copied & pasted function rather than the importing from embedding_utils. Let me know if this solves your issue.

[localjo]

Thanks for your response. I think the issue was that, though it looked like I had credit in my account, I was still using the introductory credit while trying to learn how to use the API for my purposes. I upgraded my account to a paid account and then didn't have any issues. Part of me found it frustrating that I would have to upgrade just so that I could try a demo, just so I could see how the embeddings API could be used for semantic text search. But I can also see why rate limiting the introductory credit is a good idea. I think the solution here would be to have better documentation of the suggested replacement of the search API functionality. It would be helpful to have language agnostic documentation that explains which API calls are needed to replace the search API, in such a way that I could make those calls with curl or an SDK in my preferred language and get the same results that I would have gotten from the search API. I shouldn't have to set up a development environment in a language I'm unfamiliar with, install a demo, and debug getting it to run just to understand the recommended replacement for what was previously a simple API with clear documentation.

[ted-at-openai]

I've added some more rate limit documentation here: https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb

Hope it helps you, or others who find themselves in the same spot in the future.