Retrial Mode #469
Description
Describe the feature or improvement you're requesting
Hello, I am currently maintaining AutoGPT and we have a lot of errors that we handle manually on the client side.
Did you consider providing a retrial option to abstract all this away from the user ? Pretty much everyone is building an exponential backoff and adds retrials to catch the dozens of errors OpenAI issues when you query them too much.
Less time building that = more time spending openai tokens :)
It could be an argument called "retrials", you can add it incrementally to the most important methods first and you can also include the exponential backoff in there to avoid abuse.
There is a ton of errors that OpenAI issues and that are not "real errors" and should lead to the client retrying.
Examples above.
Additional context
here is one example, this is a 500, but it should definitely be retried:
openai.error.APIError: The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 1aa6b47fe412ec21d7cbeff95babf6d4 in your message.) {
"error": {
"message": "The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 1aa6b47fe412ec21d7cbeff95babf6d4 in your message.)",
"type": "server_error",
"param": null,
"code": null
}
}
and there are some 408 as well.