[Enhancement]: Implement timeout feature for GPT4 API

[BytePhoenixCoding]

Sometimes when running it, the API hangs and gets stuck on 'waiting'.

I would recommend a feature where if there are no new tokens generated from the API (ie no new response) in X amount of seconds, it re-completes the failed development step

[senko]

A simple way of achieving this would be to add timeout=X (where X is some value in seconds, eg. 120, ie. 2 minutes) to fail the request if it doesn't complete within a reasonable amount of time:

gpt-pilot/pilot/utils/llm_connection.py

Line 343 in a6bd846

response = requests.post(

This is not perfect but would probably be better than the current solution (wait forever).

[dsitnik]

@senko tried it with timeout=120. Still gets stuck on waiting.

[senko]

@senko tried it with timeout=120. Still gets stuck on waiting.

Could you post a screenshot or copy/paste (if there's anything to copy paste)? If you're using CLI, you can also hit Ctrl-C to stop it, which wil also give you a full exception stack trace (~50 lines of messages), if you can copy/paste that it would also help out in pinpointing the location of the wait.

[dsitnik]

@senko Here it is:

^C? We would appreciate if you let us store your initial app prompt. If you are OK with that, please just press ENTER                                                                     
Traceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 88, in <module>
    exit_gpt_pilot(project, ask_feedback)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 102, in exit_gpt_pilot
    ask_to_store_prompt(project, path_id)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 63, in ask_to_store_prompt
    answer = styled_text(project, question, ignore_user_input_count=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/questionary.py", line 35, in styled_text
    response = questionary.text(question, style=used_style).unsafe_ask()  # .ask() is included here
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/questionary/question.py", line 92, in unsafe_ask
    return self.application.run()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 973, in run
    return loop.run_until_complete(coro)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 607, in run_forever
    self._run_once()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1922, in _run_once
    handle._run()
  File "/usr/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 881, in run_async
    return await _run_async(f)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 744, in _run_async
    result = await f
             ^^^^^^^
KeyboardInterrupt

[dsitnik]

@senko and this is the output I get when I change the model to gpt-3.5-turbo in .env

^CTraceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 77, in <module>
    project.start()
  File "/home/dario/gpt-pilot/pilot/helpers/Project.py", line 141, in start
    self.developer.start_coding()
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 66, in start_coding
    self.implement_task(i, dev_task)
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 113, in implement_task
    result = self.execute_task(convo_dev_task,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 405, in execute_task
    result = self.step_code_change(convo, task_description, step, i, test_after_code_changes)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 163, in step_code_change
    updated_convo = code_monkey.implement_code_changes(convo, task_description, step['code_change_description'],
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/CodeMonkey.py", line 22, in implement_code_changes
    llm_response = convo.send_message('development/implement_changes.prompt', {
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/AgentConvo.py", line 86, in send_message
    response = create_gpt_chat_completion(self.messages, self.high_level_step, self.agent.project,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 106, in create_gpt_chat_completion
    response = stream_gpt_completion(gpt_data, req_type, project)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 172, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 343, in stream_gpt_completion
    response = requests.post(
               ^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
                       ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1378, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1311, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1167, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 88, in <module>
    exit_gpt_pilot(project, ask_feedback)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 100, in exit_gpt_pilot
    send_telemetry(path_id)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 20, in send_telemetry
    response = requests.post("https://api.pythagora.io/telemetry", json=telemetry_data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
KeyboardInterrupt

[dsitnik]

I noticed that it often gets stuck on "Implementing code changes for". For example:

Implementing code changes for e `Create a Django data migration to add a default Site object with domain 'localhost'. You must create a new migration file in the 'migrations' directory of an app (e.g., 'secopi_app'). Use the 'Site' model from 'django.contrib.sites.models' to create the new object.

Example inside a migration file:


from django.db import migrations
from django.conf import settings


def create_site(apps, schema_editor):
    Site = apps.get_model('sites', 'Site')
    Site.objects.create(name='localhost', domain='localhost')


class Migration(migrations.Migration):

    dependencies = [
        ('your_app_name', 'previous_migration_file_name'),
    ]

    operations = [
        migrations.RunPython(create_site),
    ]

Run the migrations after creating this file using the command 'python manage.py migrate'.`

[techjeylabs]

closing this issue since it got solved with 2 timeouts