Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add try/except on release of work Unit and add force to workunit reaper #15129

Open
wants to merge 3 commits into
base: devel
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 12 additions & 2 deletions awx/main/tasks/system.py
Original file line number Diff line number Diff line change
Expand Up @@ -684,8 +684,18 @@ def awx_receptor_workunit_reaper():
jobs_with_unreleased_receptor_units = UnifiedJob.objects.filter(work_unit_id__in=unit_ids).exclude(status__in=ACTIVE_STATES)
for job in jobs_with_unreleased_receptor_units:
logger.debug(f"{job.log_format} is not active, reaping receptor work unit {job.work_unit_id}")
receptor_ctl.simple_command(f"work cancel {job.work_unit_id}")
receptor_ctl.simple_command(f"work release {job.work_unit_id}")
try:
receptor_ctl.simple_command(f"work cancel {job.work_unit_id}")
receptor_ctl.simple_command(f"work release {job.work_unit_id}")
except Exception as e:
# force release of work unit. Try/except is required becasue force-release
# try to release the job one time and thne delete it from worklist
logger.error(f"Error on cancel or release {job.work_unit_id} with error {str(e)}.Try Force it...")
try:
receptor_ctl.simple_command(f"work force-release {job.work_unit_id}")
logger.debug(f"{job.log_format} is now released")
except Exception as e:
logger.error(f"Error on force-release {job.work_unit_id} with error {str(e)}. Skip It")

administrative_workunit_reaper(receptor_work_list)

Expand Down