ConnectionError: Error 111 connecting to 127.0.0.1:11311. Connection refused

[qiaofeng1227]

Description of the issue

I start erpnext v15.10.4 by pwd.yml, I haven't made any changes, the container status is normal, but there are some exceptions in the queue-long container log. The ERPNext web interface is also normal and can be initialized. Here is the error log:(Stacktrace)

Context information (for bug reports)

Steps to reproduce the issue

1. git clone frappe_docker
2. docker compose start pwd.yml

Observed result

Expected result

Stacktrace / full error message if available

redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 699, in connect
    sock = self.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
           ^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 700, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 1002, in _connect
    raise err
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 990, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 114, in <module>
    main()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 20, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/scheduler.py", line 208, in start_worker
    start_worker(
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 275, in start_worker
    redis_connection = get_redis_conn(username=rq_username, password=rq_password)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 502, in get_redis_conn
    return get_redis_connection_without_auth()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 524, in get_redis_connection_without_auth
    _redis_queue_conn = RedisQueue.get_connection()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/redis_queue.py", line 21, in get_connection
    conn.ping()
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/commands/core.py", line 1205, in ping
    return self.execute_command("PING", **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/client.py", line 1266, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 1457, in get_connection
    connection.connect()
  File "/home/frappe/frappe-bench/env/lib/python3.11/site-packages/redis/connection.py", line 705, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
Please make sure that Redis Queue runs @ redis://127.0.0.1:11311. Redis reported error: Error 111 connecting to 127.0.0.1:11311. Connection refused.
2024-01-24 03:56:30,231 Worker rq:worker:b19ccb29a23d4e36b45f0a69f9843b44.c05e61271a5c.1.home-frappe-frappe-bench:long,default,short started with PID 1, version 1.15.1
2024-01-24 03:56:30,232 Subscribing to channel rq:pubsub:b19ccb29a23d4e36b45f0a69f9843b44.c05e61271a5c.1.home-frappe-frappe-bench:long,default,short
2024-01-24 03:56:30,234 *** Listening on home-frappe-frappe-bench:long, home-frappe-frappe-bench:default, home-frappe-frappe-bench:short...
2024-01-24 03:56:30,234 Cleaning registries for queue: home-frappe-frappe-bench:long

[revant]

Those errors happened when queue was trying to connect to default redis host.

Once the configuration job is complete the common_site_config.json has the right redis hosts for the workers to connect to. Workers started after they found correct connection.

If you don't want to ignore the errors, you can start all frappe containers only after configuration container completes.

[github-actions]

This issue has been automatically marked as stale. You have a week to explain why you believe this is an error.

[qiaofeng1227]

Thank you for your reply. I apologize for taking so long to reply to you again.
What you mean is that this error is okay, it only appears during startup and can be ignored.
It means that queue long needs to wait for the configuration to be completed before starting to avoid errors, like this:

Dependencies on: 
    configurator: 
        Condition: service_health

If that's the case, can you add a health_check in the configurator to avoid logging errors?