Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yum update hangs with name argument '*' #51284

Closed
Timoses opened this issue Jan 24, 2019 · 23 comments
Closed

yum update hangs with name argument '*' #51284

Timoses opened this issue Jan 24, 2019 · 23 comments
Assignees
Labels
affects_2.7 This issue/PR affects Ansible v2.7 bug This issue/PR relates to a bug. has_pr This issue has an associated PR. module This issue/PR relates to a module. needs_info This issue requires further information. Please answer any outstanding questions. packaging Packaging category support:core This issue/PR relates to code supported by the Ansible Engineering Team.

Comments

@Timoses
Copy link

Timoses commented Jan 24, 2019

SUMMARY

yum module with argument name: '*' hangs indefinitely.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

yum

ANSIBLE VERSION
ansible 2.7.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/xxx/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/xxx/.local/lib/python2.7/site-packages/ansible
  executable location = /home/xxx/.local/bin/ansible
  python version = 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]
CONFIGURATION
DEFAULT_LOG_PATH(/home/xxx/test/ansible.cfg) = /home/xxx/test/log/ansible.log
OS / ENVIRONMENT

Operating System: CentOS Linux 7 (Core)
ethtool -i eth0
driver: virtio_net
version: 1.0.0

STEPS TO REPRODUCE
- hosts: kubernetes-03
  tasks:
    - yum:
       name: '*'
EXPECTED RESULTS

Ansible finishes and succeeds or reports failure message.

ACTUAL RESULTS

Running with ansible-playbook -vvv --become --become-user=root -i inventory/hosts.ini playbook.yml:

<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"'' 
<kubernetes-03> (0, '/home/xxx\n', '') 
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221 `" && echo ansible-tmp-1548343325.74-169351418223221="` echo /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221 `" ) && sleep 0'"'"'' 
<kubernetes-03> (0, 'ansible-tmp-1548343325.74-169351418223221=/home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221\n', '')
Using module file /home/xxx/.local/lib/python2.7/site-packages/ansible/modules/packaging/os/yum.py
<kubernetes-03> PUT /home/xxx/.ansible/tmp/ansible-local-14008lcBKM9/tmpQ5LoEm TO /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221/AnsiballZ_yum.py
<kubernetes-03> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf '[kubernetes-03]'
<kubernetes-03> (0, 'sftp> put /home/xxx/.ansible/tmp/ansible-local-14008lcBKM9/tmpQ5LoEm /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221/AnsiballZ_yum.py\n', '')
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'chmod u+x /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221/ /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221/AnsiballZ_yum.py && sleep 0'"'"''
<kubernetes-03> (0, '', '')
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf -tt kubernetes-03 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-mryofwozpjczmlkcjdlmumvqbtmeawrn; /usr/bin/python /home/xxx/.ansible/tmp/ansible-tmp-1548343325.74-169351418223221/AnsiballZ_yum.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded

I can simply log onto the machine and do sudo yum -y update and it works.

The yum repository used is a privately hosted one.

@ansibot
Copy link
Contributor

ansibot commented Jan 24, 2019

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot
Copy link
Contributor

ansibot commented Jan 24, 2019

@ansibot ansibot added affects_2.7 This issue/PR affects Ansible v2.7 bug This issue/PR relates to a bug. module This issue/PR relates to a module. needs_triage Needs a first human triage before being processed. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Jan 24, 2019
@Timoses
Copy link
Author

Timoses commented Jan 24, 2019

Here is the same with ANSIBLE_DEBUG=1

 18004 1548347166.00660:  ^ task is: TASK: try yum update
 18004 1548347166.00669:  ^ state is: HOST STATE: block=2, task=2, rescue=0, always=0, run_state=ITERATING_TASKS, fail_state=FAILED_NONE, pending_setup=False, tasks child state? (None), rescue child state? (None), always child state? (None), did rescue? False, did start at task? False
 18004 1548347166.00679: done building task lists
 18004 1548347166.00687: counting tasks in each state of execution
 18004 1548347166.00696: done counting tasks in each state of execution:
	num_setups: 0
	num_tasks: 1
	num_rescue: 0
	num_always: 0
 18004 1548347166.00704: advancing hosts in ITERATING_TASKS
 18004 1548347166.00712: starting to advance hosts
 18004 1548347166.00720: getting the next task for host kubernetes-03
 18004 1548347166.00729: done getting next task for host kubernetes-03
 18004 1548347166.00737:  ^ task is: TASK: try yum update
 18004 1548347166.00745:  ^ state is: HOST STATE: block=2, task=2, rescue=0, always=0, run_state=ITERATING_TASKS, fail_state=FAILED_NONE, pending_setup=False, tasks child state? (None), rescue child state? (None), always child state? (None), did rescue? False, did start at task? False
 18004 1548347166.00754: done advancing hosts to next task
 18004 1548347166.00791: Loading ActionModule 'yum' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/action/yum.py (found_in_cache=False, class_only=True)
 18004 1548347166.00803: getting variables
 18004 1548347166.00812: in VariableManager get_vars()
 18004 1548347166.00852: Loading TestModule 'core' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/core.py (found_in_cache=True, class_only=False)
 18004 1548347166.00863: Loading TestModule 'files' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/files.py (found_in_cache=True, class_only=False)
 18004 1548347166.00872: Loading TestModule 'mathstuff' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/mathstuff.py (found_in_cache=True, class_only=False)
 18004 1548347166.00911: Loading FilterModule 'core' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/core.py (found_in_cache=True, class_only=False)
 18004 1548347166.00922: Loading FilterModule 'ipaddr' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/ipaddr.py (found_in_cache=True, class_only=False)
 18004 1548347166.00932: Loading FilterModule 'json_query' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/json_query.py (found_in_cache=True, class_only=False)
 18004 1548347166.00941: Loading FilterModule 'mathstuff' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/mathstuff.py (found_in_cache=True, class_only=False)
 18004 1548347166.00951: Loading FilterModule 'network' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/network.py (found_in_cache=True, class_only=False)
 18004 1548347166.00959: Loading FilterModule 'urlsplit' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/urlsplit.py (found_in_cache=True, class_only=False)
 18004 1548347166.01042: Calling all_inventory to load vars for kubernetes-03
 18004 1548347166.01055: Calling groups_inventory to load vars for kubernetes-03
 18004 1548347166.01066: Calling all_plugins_inventory to load vars for kubernetes-03
 18004 1548347166.01088: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.01112: Loading data from /home/xxx/code/xxx_kubespray_tests/inventory/group_vars/all/kubespray.yml
 18004 1548347166.01123: Calling all_plugins_play to load vars for kubernetes-03
 18004 1548347166.01142: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.01162: Calling groups_plugins_inventory to load vars for kubernetes-03
 18004 1548347166.01181: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.01204: Calling groups_plugins_play to load vars for kubernetes-03
 18004 1548347166.01223: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.01258: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.01287: Loading VarsModule 'host_group_vars' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/vars/host_group_vars.py (found_in_cache=True, class_only=False)
 18004 1548347166.03326: done with get_vars()
 18004 1548347166.03352: done getting variables
 18004 1548347166.03365: sending task start callback, copying the task so we can template it temporarily
 18004 1548347166.03374: done copying, going to template now
 18004 1548347166.03383: done templating
 18004 1548347166.03392: here goes the callback...

TASK [try yum update] ********************************************************************************************************************************************************************************************************************************************************************
task path: /home/xxx/code/xxx_kubespray_tests/playbook.yml:21
 18004 1548347166.03421: sending task start callback
 18004 1548347166.03430: entering _queue_task() for kubernetes-03/yum
 18004 1548347166.03560: worker is 1 (out of 1 available)
 18004 1548347166.03635: exiting _queue_task() for kubernetes-03/yum
 18004 1548347166.03658: done queuing things up, now waiting for results queue to drain
 18004 1548347166.03675: waiting for pending results...
 18115 1548347166.03995: running TaskExecutor() for kubernetes-03/TASK: try yum update
 18115 1548347166.04089: in run() - task 006097ae-f99c-8b70-bc0c-00000000000e
 18115 1548347166.04173: calling self._execute()
 18115 1548347166.04844: Loading Connection 'ssh' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py
 18115 1548347166.04983: Loading ShellModule 'sh' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/shell/sh.py
 18115 1548347166.05121: assigned :doc
 18115 1548347166.05161: Loading ModuleDocFragment 'shell_common' from /home/xxx/.local/lib/python2.7/site-packages/ansible/utils/module_docs_fragments/shell_common.py
 18115 1548347166.05350: Loaded config def from plugin (shell/sh)
 18115 1548347166.05371: Loading ShellModule 'sh' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/shell/sh.py (found_in_cache=True, class_only=False)
 18115 1548347166.09433: assigned :doc
 18115 1548347166.09524: Loaded config def from plugin (connection/ssh)
 18115 1548347166.09672: Loading ActionModule 'yum' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/action/yum.py (found_in_cache=True, class_only=False)
 18115 1548347166.09685: starting attempt loop
 18115 1548347166.09695: running the handler
 18115 1548347166.09730: Loading TestModule 'core' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/core.py (found_in_cache=True, class_only=False)
 18115 1548347166.09740: Loading TestModule 'files' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/files.py (found_in_cache=True, class_only=False)
 18115 1548347166.09750: Loading TestModule 'mathstuff' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/test/mathstuff.py (found_in_cache=True, class_only=False)
 18115 1548347166.09788: Loading FilterModule 'core' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/core.py (found_in_cache=True, class_only=False)
 18115 1548347166.09799: Loading FilterModule 'ipaddr' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/ipaddr.py (found_in_cache=True, class_only=False)
 18115 1548347166.09808: Loading FilterModule 'json_query' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/json_query.py (found_in_cache=True, class_only=False)
 18115 1548347166.09821: Loading FilterModule 'mathstuff' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/mathstuff.py (found_in_cache=True, class_only=False)
 18115 1548347166.09830: Loading FilterModule 'network' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/network.py (found_in_cache=True, class_only=False)
 18115 1548347166.09840: Loading FilterModule 'urlsplit' from /home/xxx/.local/lib/python2.7/site-packages/ansible/plugins/filter/urlsplit.py (found_in_cache=True, class_only=False)
 18115 1548347166.10085: _low_level_execute_command(): starting
 18115 1548347166.10102: _low_level_execute_command(): executing: /bin/sh -c 'echo ~ && sleep 0'
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
 18115 1548347166.13172: stdout chunk (state=2):
>>>/home/xxx
<<<
 18115 1548347166.13277: stderr chunk (state=3):
>>><<<
 18115 1548347166.13298: stdout chunk (state=3):
>>><<<
<kubernetes-03> (0, '/home/xxx\n', '')
 18115 1548347166.13336: _low_level_execute_command() done: rc=0, stdout=/home/xxx
, stderr=
 18115 1548347166.13358: _low_level_execute_command(): starting
 18115 1548347166.13377: _low_level_execute_command(): executing: /bin/sh -c '( umask 77 && mkdir -p "` echo /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121 `" && echo ansible-tmp-1548347166.1-22512381894121="` echo /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121 `" ) && sleep 0'
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121 `" && echo ansible-tmp-1548347166.1-22512381894121="` echo /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121 `" ) && sleep 0'"'"''
 18115 1548347166.16850: stdout chunk (state=2):
>>>ansible-tmp-1548347166.1-22512381894121=/home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121
<<<
 18115 1548347166.16985: stderr chunk (state=3):
>>><<<
 18115 1548347166.17019: stdout chunk (state=3):
>>><<<
<kubernetes-03> (0, 'ansible-tmp-1548347166.1-22512381894121=/home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121\n', '')
 18115 1548347166.17076: _low_level_execute_command() done: rc=0, stdout=ansible-tmp-1548347166.1-22512381894121=/home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121
, stderr=
 18115 1548347166.17179: ANSIBALLZ: Using lock for yum
 18115 1548347166.17199: ANSIBALLZ: Acquiring lock
 18115 1548347166.17220: ANSIBALLZ: Lock acquired: 140598534952080
 18115 1548347166.17238: ANSIBALLZ: Creating module
 18115 1548347166.33243: ANSIBALLZ: Writing module
 18115 1548347166.33279: ANSIBALLZ: Renaming module
 18115 1548347166.33290: ANSIBALLZ: Done creating module
Using module file /home/xxx/.local/lib/python2.7/site-packages/ansible/modules/packaging/os/yum.py
 18115 1548347166.33396: transferring module to remote /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py
<kubernetes-03> PUT /home/xxx/.ansible/tmp/ansible-local-18004jWH00J/tmpGwQsQs TO /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py
<kubernetes-03> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf '[kubernetes-03]'
 18115 1548347166.33967: Sending initial data
 18115 1548347166.34001: Sent initial data (153 bytes)
 18115 1548347166.36779: stdout chunk (state=3):
>>>sftp> put /home/xxx/.ansible/tmp/ansible-local-18004jWH00J/tmpGwQsQs /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py
<<<
 18115 1548347166.38525: stderr chunk (state=3):
>>><<<
 18115 1548347166.38559: stdout chunk (state=3):
>>><<<
<kubernetes-03> (0, 'sftp> put /home/xxx/.ansible/tmp/ansible-local-18004jWH00J/tmpGwQsQs /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py\n', '')
 18115 1548347166.38613: done transferring module to remote
 18115 1548347166.38662: _low_level_execute_command(): starting
 18115 1548347166.38682: _low_level_execute_command(): executing: /bin/sh -c 'chmod u+x /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/ /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py && sleep 0'
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf kubernetes-03 '/bin/sh -c '"'"'chmod u+x /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/ /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py && sleep 0'"'"''
 18115 1548347166.42113: stderr chunk (state=2):
>>><<<
 18115 1548347166.42173: stdout chunk (state=2):
>>><<<
<kubernetes-03> (0, '', '')
 18115 1548347166.42227: _low_level_execute_command() done: rc=0, stdout=, stderr=
 18115 1548347166.42248: _low_level_execute_command(): starting
 18115 1548347166.42269: _low_level_execute_command(): using become for this command
 18115 1548347166.42312: _low_level_execute_command(): executing: /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy; /usr/bin/python /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py'"'"' && sleep 0'
<kubernetes-03> ESTABLISH SSH CONNECTION FOR USER: None
<kubernetes-03> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/af1f7b99cf -tt kubernetes-03 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy; /usr/bin/python /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
 18115 1548347166.42993: Initial state: awaiting_escalation: BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy
 18115 1548347166.47052: stdout chunk (state=1):
>>>BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy
<<<
 18115 1548347166.47092: become_success: (source=stdout, state=awaiting_escalation): 'BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy'
Escalation succeeded

I also tried logging in manually via ssh and executing /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-qchvhzmwflnnkcernioqdaxtpjrwznzy; /usr/bin/python /home/xxx/.ansible/tmp/ansible-tmp-1548347166.1-22512381894121/AnsiballZ_yum.py'"'"' && sleep 0' which also hangs.
Also the following yields the same:

$ sudo -H -S -n -u root /bin/sh
sh-4.2# /usr/bin/python /home/funke_t1/.ansible/tmp/ansible-tmp-1548346633.77-194843557357994/AnsiballZ_yum.py

@Timoses
Copy link
Author

Timoses commented Jan 24, 2019

The command that hangs is executed in https://github.com/ansible/ansible/blob/v2.7.6/lib/ansible/modules/packaging/os/yum.py#L641

The following reproduces the freezing effect:
sudo /usr/bin/repoquery --show-duplicates --plugins --disablerepo '' --enablerepo '' --qf '%{epoch}:%{name}-%{version}-%{release}.%{arch}' --whatprovides '*'

Edit: The command above actually finished after quite a while (1-2 minutes).
I stopped debugging the yum module for now. Instead see below reference to kubespray.

It would still be nice if the yum module handled such "erroneous" configurations so the user is not confronted with a frozen process.

@Timoses
Copy link
Author

Timoses commented Jan 29, 2019

The following line seems to be causing the issue:
https://github.com/ansible/ansible/blob/v2.7.6/lib/ansible/modules/packaging/os/yum.py#L976
In my case it is called with 75525 items in pkglist. It does not seem to finish ever.

@maxamillion
Copy link
Contributor

@Timoses I don't have a system available to me that I'm able to reproduce this with, is there a certain set of public yum repositories you're using that adds up to that large of a content set for pkglist that I could use to replicate the situation? Or do you otherwise have a minimal reproducer? Thank you.

needs_info

@maxamillion maxamillion self-assigned this Jan 29, 2019
@ansibot ansibot added needs_info This issue requires further information. Please answer any outstanding questions. and removed needs_triage Needs a first human triage before being processed. labels Jan 29, 2019
@Timoses
Copy link
Author

Timoses commented Jan 29, 2019

@maxamillion The host is served by a rpm repository(s) managed by spacewalk. That means there are several "channels". I'm afraid I don't have any more information, at least not as insight I can give you as I myself am not well versed with yum and rpm.
However, if you need any specific details let me know.

@ansibot ansibot removed the needs_info This issue requires further information. Please answer any outstanding questions. label Jan 29, 2019
@jborean93
Copy link
Contributor

@Testicule is sounds like you are just invoking the module without specifying the args json file that contain the arguments. If no argument is specified it will try and read from stdin. You can read all about how to debug/invoke modules outside of Ansible at https://docs.ansible.com/ansible/latest/dev_guide/debugging.html.

@Timoses
Copy link
Author

Timoses commented Jan 30, 2019

@Testicule
As your issue does not seem to be directly related to this issue (argument name: '*' causes the issue here), could you please create another issue?

@maxamillion
Copy link
Contributor

@Timoses no worries, I'll try and see if we can sort it out. I'm curious how many repos you have listed in yum repolist all and how many are marked enabled, thank you!

needs_info

@ansibot ansibot added the needs_info This issue requires further information. Please answer any outstanding questions. label Jan 30, 2019
@Timoses
Copy link
Author

Timoses commented Feb 5, 2019

@maxamillion Sorry for the delay.
yum repolist all returns

repo id        repo name         status
xxx            xxx               enabled:    42
xxx            xxx               enabled:  1553
xxx            xxx               enabled: 29441
xxx            xxx               enabled: 12722
xxx            xxx               enabled:    28
xxx            xxx               enabled:   184
xxx            xxx               enabled: 40452
xxx            xxx               enabled:   308
xxx            xxx               enabled:    75

(blanked out the channel names with xxx)

@ansibot ansibot removed the needs_info This issue requires further information. Please answer any outstanding questions. label Feb 5, 2019
@maxamillion
Copy link
Contributor

@Timoses no worries, thank you!

Out of curiosity, if you just run a yum update on that system where you're attempting ansible $that_system -m yum -a 'name="*"', do you have any metrics on the amount of time it takes just the yum command line to chew through that many repos and packages? Similar question for something like repoquery bash.

Thanks again.

needs_info

@ansibot ansibot added the needs_info This issue requires further information. Please answer any outstanding questions. label Feb 5, 2019
@Timoses
Copy link
Author

Timoses commented Feb 6, 2019

@maxamillion
I debugged the python module a bit on the host previously:

Running yum update on an outdated host took about 7 minutes (query: couple of seconds, 297MB download: 20 seconds, 595 steps (updating/installing/cleanup): rest of time).

Upon running yum update again it finishes almost within a second.

With ansible <host> -i hosts -m yum -a "name='*'" -vvv -b --become-user=root (host being the same host as above -> already updated): I waited for an hour. Ansible output:

  ansible 2.7.6
    config file = /home/xxx/ansible/ansible-kubernetes/kubespray/ansible.cfg
      configured module search path = [u'/home/xxx/ansible/ansible-kubernetes/kubespray/library
      ']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
    executable location = /usr/bin/ansible
      python version = 2.7.15 (default, Jan 10 2019, 23:20:52) [GCC 8.2.1 20181127]
      Using /home/xxx/ansible/ansible-kubernetes/kubespray/ansible.cfg as config file
      /home/xxx/ansible/ansible-kubernetes/inventory/hosts.ini did not meet host_list requirements, check plugin documentation if this is unexpected
      /home/xxx/ansible/ansible-kubernetes/inventory/hosts.ini did not meet script requirements, check plugin documentation if this is unexpected
      /home/xxx/ansible/ansible-kubernetes/inventory/hosts.ini did not meet yaml requirements, check plugin documentation if this is unexpected
      Parsed /home/xxx/ansible/ansible-kubernetes/inventory/hosts.ini inventory source with ini plugin
      META: ran handlers
      Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
      <kubernetes-dev-4> ESTABLISH SSH CONNECTION FOR USER: None
      <kubernetes-dev-4> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/25a3005c2e kubernetes-dev-4 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-isyyofaijshfqvtrmcezjgzjahnhwhvv; /usr/bin/python'"'"'"'"'"'"'"'"' && sleep 0'"'"''
      Escalation succeeded
      <kubernetes-dev-4> (0, '\n{"invocation": {"module_args": {"filter": "ansible_pkg_mgr", "gather_subset": ["!all"], "fact_path": "/etc/ansible/facts.d", "gather_timeout": 10}}, "ansible_facts": {"ansible_pkg_mgr": "yum"}}\n', '')
      Using module file /usr/lib/python2.7/site-packages/ansible/modules/packaging/os/yum.py
      <kubernetes-dev-4> ESTABLISH SSH CONNECTION FOR USER: None
      <kubernetes-dev-4> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/xxx/.ansible/cp/25a3005c2e kubernetes-dev-4 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-kuwbkqxcygzvonjqabdodswllysszndn; /usr/bin/python'"'"'"'"'"'"'"'"' && sleep 0'"'"''
      Escalation succeeded

Nothing happens (at least not visible on the ansible controller) after the second Escalation succeeded.

/etc/yum.conf (in case it helps?)

[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
plugins=1
pkgpolicy=newest
retries=20
installonly_limit=2
reposdir=/tmp

(Is there any way to print the commands that an ansible module executes? This would be useful for reproducing issues.. It's a pain debugging the python modules live : /.)

@ansibot ansibot removed the needs_info This issue requires further information. Please answer any outstanding questions. label Feb 6, 2019
@maxamillion
Copy link
Contributor

@Timoses this is great, thank you! I'll dig in and see if I can sort out the root cause.

@ansibot ansibot added the packaging Packaging category label Feb 14, 2019
@Timoses
Copy link
Author

Timoses commented May 8, 2019

@maxamillion : Any updates on this?

The documentation of the yum module is still striking me as a bit odd. Under the field name it states:

When using state=latest, this can be '*' which means run yum -y update.

It does not say that state may be anything else. Does this mean name: '*' should not be used with any other state?

Please also see the discussion here on the application of a use case with name: '*' and state: present.

The problem still persists, also when explicitly setting state: present.

@maxamillion
Copy link
Contributor

@Timoses unfortunately not, I've been under water with other priorities ... I haven't forgotten about it, but I've just not had a chance to circle back to it. Apologies.

@AechLyons
Copy link

was this ever addressed. Having the exact same issue all of a sudden

@maxamillion
Copy link
Contributor

I actually think this issue was related to #63713 ... I suspect what's happening is that it's attempting to re-instantiate the YumBase (and subsequent sack load, which is an extremely expensive operation) too many times on a large package count and timing out. That fix has been merged and there are backport pull requests open so I would hope to see this fixed in the next releases of both Ansible 2.8 and Ansible 2.9. Please let me know if you have a chance to test and verify.

@ses-team
Copy link

ses-team commented Mar 4, 2020

I am having the same issue in Ansible v2.9.2 using:

  • name: Install updates
    yum:
    name: ""
    state: latest
    update_cache: yes
    exclude:
    - python

    - java*

Any update on a resolution to this?

@ansibot
Copy link
Contributor

ansibot commented May 16, 2020

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

@maxamillion
Copy link
Contributor

@ses-team I need more information about the system so that I can reproduce and diagnose.

  1. What distro and release?
  2. What state is the system in? (fresh install, pre-existing, container, other?)
  3. What repos are enabled/disabled?
  4. How long does the corresponding action take from the command line?

needs_info

@ansibot ansibot added the needs_info This issue requires further information. Please answer any outstanding questions. label Jun 17, 2020
@ansibot
Copy link
Contributor

ansibot commented Jul 22, 2020

@Timoses This issue is waiting for your response. Please respond or the issue will be closed.

click here for bot help

@ansibot
Copy link
Contributor

ansibot commented Aug 23, 2020

@Timoses You have not responded to information requests in this issue so we will assume it no longer affects you. If you are still interested in this, please create a new issue with the requested information.

click here for bot help

@ansibot ansibot closed this as completed Aug 23, 2020
@ansible ansible locked and limited conversation to collaborators Sep 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.7 This issue/PR affects Ansible v2.7 bug This issue/PR relates to a bug. has_pr This issue has an associated PR. module This issue/PR relates to a module. needs_info This issue requires further information. Please answer any outstanding questions. packaging Packaging category support:core This issue/PR relates to code supported by the Ansible Engineering Team.
Projects
None yet
Development

No branches or pull requests

7 participants
@maxamillion @ansibot @Timoses @jborean93 @AechLyons @ses-team and others