Support timeout and retry

[yinggoga]

openai always timeout or raise exception, is there a plan to support openai request timeout and retry?

[niklub]

Hi, @yinggoga ! Thanks for your report.
Could you please provide me with your setup? That would be helpful to understand and catch the errors in a specific Runtime you use.
Meanwhile - adding support for OpenAIChatRuntime in this PR #44

[yinggoga]

Hi, @yinggoga ! Thanks for your report. Could you please provide me with your setup? That would be helpful to understand and catch the errors in a specific Runtime you use. Meanwhile - adding support for OpenAIChatRuntime in this PR #44

Thanks!
I use OpenAIChatRuntime and One of the errors is:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/openai/api_requestor.py", line 765, in _interpret_response_line
    data = json.loads(rbody)
  File "/usr/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/mnt/data/LLM_code/auto_prompt/LMOps-main/adala_agent/api_binary.py", line 97, in <module>
    main()
  File "/mnt/data/LLM_code/auto_prompt/LMOps-main/adala_agent/api_binary.py", line 93, in main
    agent.learn(learning_iterations=3, accuracy_threshold=0.99)
  File "/usr/local/lib/python3.9/dist-packages/adala/agents/base.py", line 268, in learn
    predictions = self.skills.apply(inputs, runtime=runtime)
  File "/usr/local/lib/python3.9/dist-packages/adala/skills/skillset.py", line 173, in apply
    skill_output = skill.apply(skill_input, runtime)
  File "/usr/local/lib/python3.9/dist-packages/adala/skills/_base.py", line 121, in apply
    return runtime.batch_to_batch(
  File "/usr/local/lib/python3.9/dist-packages/adala/runtimes/base.py", line 80, in batch_to_batch
    output = batch.progress_apply(
  File "/usr/local/lib/python3.9/dist-packages/tqdm/std.py", line 920, in inner
    return getattr(df, df_function)(wrapper, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/frame.py", line 10037, in apply
    return op.apply().__finalize__(self, method="apply")
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/apply.py", line 831, in apply
    return self.apply_standard()
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/apply.py", line 957, in apply_standard
    results, res_index = self.apply_series_generator()
  File "/usr/local/lib/python3.9/dist-packages/pandas/core/apply.py", line 973, in apply_series_generator
    results[i] = self.func(v, *self.args, **self.kwargs)
  File "/usr/local/lib/python3.9/dist-packages/tqdm/std.py", line 915, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/adala/runtimes/_openai.py", line 139, in record_to_record
    completion_text = self.execute(messages)
  File "/usr/local/lib/python3.9/dist-packages/adala/runtimes/_openai.py", line 88, in execute
    completion = openai.ChatCompletion.create(
  File "/usr/local/lib/python3.9/dist-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/openai/api_resources/abstract/engine_api_resource.py", line 155, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.9/dist-packages/openai/api_requestor.py", line 299, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.9/dist-packages/openai/api_requestor.py", line 710, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.9/dist-packages/openai/api_requestor.py", line 767, in _interpret_response_line
    raise error.APIError(
openai.error.APIError: HTTP code 524 from API (<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>


<title>openkey.cloud | 524: A timeout occurred</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/main.css" />


</head>
<body>
<div id="cf-wrapper">
    <div id="cf-error-details" class="p-0">
        <header class="mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8">
            <h1 class="inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2">
              <span class="inline-block">A timeout occurred</span>
              <span class="code-label">Error code 524</span>
            </h1>
            <div>
               Visit <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=openkey.cloud" target="_blank" rel="noopener noreferrer">cloudflare.com</a> for more information.
            </div>
            <div class="mt-3">2023-11-28 13:11:17 UTC</div>
        </header>
        <div class="my-8 bg-gradient-gray">
            <div class="w-240 lg:w-full mx-auto">
                <div class="clearfix md:px-8">

<div id="cf-browser-status" class=" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">

    <span class="cf-icon-browser block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>

  </div>
  <span class="md:block w-full truncate">You</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">

    Browser

  </h3>
  <span class="leading-1.3 text-2xl text-green-success">Working</span>
</div>

<div id="cf-cloudflare-status" class=" relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">
    <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=openkey.cloud" target="_blank" rel="noopener noreferrer">
    <span class="cf-icon-cloud block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>
    </a>
  </div>
  <span class="md:block w-full truncate">San Jose</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">
    <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=openkey.cloud" target="_blank" rel="noopener noreferrer">
    Cloudflare
    </a>
  </h3>
  <span class="leading-1.3 text-2xl text-green-success">Working</span>
</div>

<div id="cf-host-status" class="cf-error-source relative w-1/3 md:w-full py-15 md:p-0 md:py-8 md:text-left md:border-solid md:border-0 md:border-b md:border-gray-400 overflow-hidden float-left md:float-none text-center">
  <div class="relative mb-10 md:m-0">

    <span class="cf-icon-server block md:hidden h-20 bg-center bg-no-repeat"></span>
    <span class="cf-icon-error w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>

  </div>
  <span class="md:block w-full truncate">openkey.cloud</span>
  <h3 class="md:inline-block mt-3 md:mt-0 text-2xl text-gray-600 font-light leading-1.3">

    Host

  </h3>
  <span class="leading-1.3 text-2xl text-red-error">Error</span>
</div>

                </div>
            </div>
        </div>

        <div class="w-240 lg:w-full mx-auto mb-8 lg:px-8">
            <div class="clearfix">
                <div class="w-1/2 md:w-full float-left pr-6 md:pb-10 md:pr-0 leading-relaxed">
                    <h2 class="text-3xl font-normal leading-1.3 mb-4">What happened?</h2>
                    <p>The origin web server timed out responding to this request.</p>
                </div>
                <div class="w-1/2 md:w-full float-left leading-relaxed">
                    <h2 class="text-3xl font-normal leading-1.3 mb-4">What can I do?</h2>
                          <h3 class="text-15 font-semibold mb-2">If you're a visitor of this website:</h3>
      <p class="mb-6">Please try again in a few minutes.</p>

      <h3 class="text-15 font-semibold mb-2">If you're the owner of this website:</h3>
      <p><span>The connection to the origin web server was made, but the origin web server timed out before responding. The likely cause is an overloaded background task, database or application, stressing the resources on your web server. To resolve, please work with your hosting provider or web development team to free up resources for your database or overloaded application.</span> <a rel="noopener noreferrer" href="https://support.cloudflare.com/hc/en-us/articles/200171926-Error-524">Additional troubleshooting information here.</a></p>
                </div>
            </div>
        </div>

        <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300">
  <p class="text-13">
    <span class="cf-footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">82d2d8ae78589435</strong></span>
    <span class="cf-footer-separator sm:hidden">&bull;</span>
    <span id="cf-footer-item-ip" class="cf-footer-item hidden sm:block sm:mb-1">
      Your IP:
      <button type="button" id="cf-footer-ip-reveal" class="cf-footer-ip-reveal-btn">Click to reveal</button>
      <span class="hidden" id="cf-footer-ip">123.58.64.57</span>
      <span class="cf-footer-separator sm:hidden">&bull;</span>
    </span>
    <span class="cf-footer-item sm:block sm:mb-1"><span>Performance &amp; security by</span> <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_524&utm_campaign=openkey.cloud" id="brand_link" target="_blank">Cloudflare</a></span>

  </p>
  <script>(function(){function d(){var b=a.getElementById("cf-footer-item-ip"),c=a.getElementById("cf-footer-ip-reveal");b&&"classList"in b&&(b.classList.remove("hidden"),c.addEventListener("click",function(){c.classList.add("hidden");a.getElementById("cf-footer-ip").classList.remove("hidden")}))}var a=document;document.addEventListener&&a.addEventListener("DOMContentLoaded",d)})();</script>
</div><!-- /.error-footer -->


    </div>
</div>
</body>
</html>
)

[niklub]

Ok, I see. Have you tried to run the same thing on the latest release pip install adala==0.0.4 ?

[yinggoga]

Ok, I see. Have you tried to run the same thing on the latest release pip install adala==0.0.4 ?

Sure，I'm going to try it out in the next few days

[yinggoga]

Ok, I see. Have you tried to run the same thing on the latest release pip install adala==0.0.4 ?

It worked!Thanks~