Skip to content
This repository was archived by the owner on Apr 1, 2025. It is now read-only.

Commit d9004cf

Browse files
committed
README: document how to purge ci_fetch queue
Signed-off-by: Charles Oliveira <charles.oliveira@linaro.org>
1 parent 2978a2f commit d9004cf

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,15 @@ Usually a command to describe what is going on on a pod is
113113
Starting celery with '--without-mingle' prevented it from crashing everytime a new worker was started in parallel.
114114
More info: https://stackoverflow.com/questions/55249197/what-are-the-consequences-of-disabling-gossip-mingle-and-heartbeat-for-celery-w
115115

116+
## Never-ending tasks in ci_fetch queue
117+
118+
Sometimes celery suffers from `celery_chord_unlock` tasks that never reach a timeout and causes the fetch-workers to
119+
do useless work. I still have not found the reason for this yet, but until then, it's convenient to purge the ci_fetch queue.
120+
121+
1. First kill all fetch-workers
122+
2. ssh -i tmp/qareports_private_ssh_key `cat terraform/generated/production_rabbitmq_host_public` (you'll need to run `./qareports production queues` first)
123+
3. sudo rabbitmqctl purge_queue ci_fetch
124+
116125
## Emails
117126

118127
We're currently using AWS Simple Email Service aka SES to send emails. On an account that SES was never used, AWS puts it under sandbox

0 commit comments

Comments
 (0)