Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

awx ad-hoc command (ping) job stay in running status indefinitely #14796

Open
5 of 11 tasks
bskou57 opened this issue Jan 23, 2024 · 2 comments
Open
5 of 11 tasks

awx ad-hoc command (ping) job stay in running status indefinitely #14796

bskou57 opened this issue Jan 23, 2024 · 2 comments

Comments

@bskou57
Copy link

bskou57 commented Jan 23, 2024

Please confirm the following

  • I agree to follow this project's code of conduct.
  • I have checked the current issues for duplicates.
  • I understand that AWX is open source software provided for free and that I might not receive a timely response.
  • I am NOT reporting a (potential) security vulnerability. (These should be emailed to security@ansible.com instead.)

Bug Summary

Hi

I run ping ad-hoc command via awx web interface with high fork

At the beginning it is running faster and finally stay in running state with forking processes 2 by 2 only

why is forking value/capacity decrease over the processing time ?

Thanks for your support

AWX version

23.5.0

Select the relevant components

  • UI
  • UI (tech preview)
  • API
  • Docs
  • Collection
  • CLI
  • Other

Installation method

kubernetes

Modifications

no

Ansible version

2.14.2

Operating system

Redhat 8.8

Web browser

Chrome

Steps to reproduce

run awx adhoc command (ping module for example) via inventory on big server list (over 4000) and never finished
Note that I have the same behavior with command module

Expected results

job completed in failed or successful status but stay in running status indefinitely.

Actual results

adhoc (ping module) command job is running indefinitely

Additional information

Nothing to highlight in pods logs or podman logs

@fosterseth
Copy link
Member

fosterseth commented Jan 24, 2024

thanks for raising this issue. how are you monitoring # of active forks at a time? Are you inferring that based on job stdout, or are you exec-ing into pod and looking at running ansible-playbook PIDs?

what fork value are you setting in the ac-hoc command?

where is this job running (i.e. remote execution node, container group, cluster automation pod, etc).

Are there CPU limits set on the cluster where this is running? How many actual CPU cores is on that k8s worker node?

also this sounds more like an ansible issue, not AWX, and you might gain traction by creating a new thread on forum.ansible.com under Get Help section

@bskou57
Copy link
Author

bskou57 commented Jan 24, 2024

Hello

I used ps command to count number of ssh connection which should be equal zo number of forks

Yes I am looking for pids

we do not have cpu limit on execution node (where the podman run) and the control node where I have awx installed with pods on k3s single cluster node

How can I check resources limite on k3s ?

ok will raise request to ansible team

Thanks for your support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants