Job doesn't run with message: "Unable to process request: No celery workers running on queue default"

[apanagio]

I have a Nautobot 1.6.7 instance running.

Approximately 5% of the time I try to start a job, I get the message Unable to process request: No celery workers running on queue default. This happens regardless of how I start the jobs, either from the UI or from the API.
I haven't found a way to reproduce it.
The only log message I get (even when running in DEBUG loglevel) is Service Unavailable: /nautobot/api/extras/jobs/c4889768-a3b3-454e-b746-fa960431793b/run/

I have 2 workers running on 1 queue, but the same thing happened when I had 1 worker.

Does anybody have any suggestions on how I could fix the issue, or maybe point me to a direction I should investigate.

thanks in advance

[gsnider2195]

The celery command that retrieves the workers on a queue has a timeout of 1 second by default so if your workers are taking longer than that to respond then you'll receive the error about no available workers. The timeout isn't configurable today in Nautobot but could be set here

[apanagio]

Thank you so much!

I did the following change

# celery_inspect = app.control.inspect()
celery_inspect = app.control.inspect(timeout=1)
active_queues = celery_inspect.active_queues()
if active_queues is None:
    celery_inspect = app.control.inspect(timeout=10)
    active_queues = celery_inspect.active_queues()

and so far it eliminates the problem