Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gather_timeout from ansible.cfg is ignored when running a playbook. #23753

Closed
i1caro opened this issue Apr 19, 2017 · 7 comments · Fixed by #23826
Closed

gather_timeout from ansible.cfg is ignored when running a playbook. #23753

i1caro opened this issue Apr 19, 2017 · 7 comments · Fixed by #23826
Labels
affects_2.3 This issue/PR affects Ansible v2.3 bug This issue/PR relates to a bug.

Comments

@i1caro
Copy link

i1caro commented Apr 19, 2017 •
edited

ISSUE TYPE
  • Bug Report
COMPONENT NAME

Timeout Module

ANSIBLE VERSION
ansible 2.3.0.0
  config file = /home/maximus/work/server-recipes/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.9 (default, Apr  2 2015, 15:33:21) [GCC 4.9.2]
CONFIGURATION
[defaults]
hostfile = hosts
remote_user = root
private_key_file = ~/.ssh/myne
host_key_checking = False
force_handlers = True
forks = 50
inventory = inventory
timeout = 10
gather_timeout = 300
OS / ENVIRONMENT

Ubuntu 15

SUMMARY

The variable gather_timeout is being ignored. This changed from Ansible 2.2 to 2.3 since after the upgrade some servers started to timeout. In order to debug I changed the ansible code to spit the following line by. Changing facts to.

GATHER_TIMEOUT=None

class TimeoutError(Exception):
    pass

def timeout(seconds=None, error_message="Timer expired"):
    if seconds is None:
        seconds = globals().get('GATHER_TIMEOUT') or 10
    def decorator(func):
        def _handle_timeout(signum, frame):
            raise TimeoutError(
                error_message +
                ' global timer %s, seconds %s' % (
                    globals().get('GATHER_TIMEOUT'),
                    seconds
                )
            )

Output:

fatal: [server.something.com]: FAILED! => {"changed": false, "cmd": "/bin/lsblk --list --noheadings --paths --output NAME,UUID --exclude 2", "failed": true, "msg": "Timer expired global timer 300, seconds 10", "rc": 257}

This means that on the instantiation of the decorator globals().get('GATHER_TIMEOUT') is either None or 10. On a brief look I would say None and then it's changed to the configuration value. I would say here:

def get_all_facts(module):

    setup_options = dict(module_setup=True)

    # Retrieve module parameters
    gather_subset = module.params['gather_subset']

    global GATHER_TIMEOUT
    GATHER_TIMEOUT = module.params['gather_timeout']

Which at this point is to late since the decorator was timeout was already set to 10.

STEPS TO REPRODUCE

Set gather_timeout in ansible.cfg

EXPECTED RESULTS

Expect timeout to match gather_timeout value.

ACTUAL RESULTS

Timeout is always 10.
@ansibot ansibot added affects_2.3 This issue/PR affects Ansible v2.3 bug_report needs_triage Needs a first human triage before being processed. labels Apr 19, 2017
@snowsky
Copy link
Contributor

snowsky commented Apr 19, 2017

Looks like the function get_mount_facts timed out.

@snowsky
Copy link
Contributor

snowsky commented Apr 19, 2017

Or probably it failed to run lsblk command. Could you run this command manually on the host?

/bin/lsblk --list --noheadings --paths --output NAME,UUID --exclude 2

@snowsky
Copy link
Contributor

snowsky commented Apr 19, 2017

It is a bug. The root cause is the variable from ansible.cfg didn't take effect, but the variable from command can:

bin/ansible localhost -m setup -a "gather_timeout=20"

@snowsky
Copy link
Contributor

snowsky commented Apr 19, 2017 •
edited

Dup #17316

@abadger abadger removed the needs_triage Needs a first human triage before being processed. label Apr 20, 2017
@abadger
Copy link
Contributor

abadger commented Apr 20, 2017

I believe @i1caro 's initial analysis is correct and this is a new bug. Looking for a fix now.

@abadger abadger mentioned this issue Apr 20, 2017
Merged
@abadger
Copy link
Contributor

abadger commented Apr 20, 2017

@i1caro Please give #23826 a try. I think I fixed the issue and added a unittest that should cover your case to keep it from regressing.

abadger added a commit to abadger/ansible that referenced this issue Apr 20, 2017
@abadger
Facts Timeout was not settable via ansible.cfg
2703661 
The timeout for gathering facts needs to be settable from three places
(highest precedence to lowest):

* programmatically
* ansible.cfg (equivalent to the user specifying it explicitly when
  calling setup)
* from the default value

The code was changed in b4bd6c8 to
allow programmatically and the default value to work correctly but
setting via ansible.cfg/parameter was broken.

This change should fix setting via ansible.cfg and adds unittests for
all three cases

Fixes ansible#23753
abadger added a commit that referenced this issue May 1, 2017
@abadger
Facts Timeout was not settable via ansible.cfg
98d223a 
The timeout for gathering facts needs to be settable from three places
(highest precedence to lowest):

* programmatically
* ansible.cfg (equivalent to the user specifying it explicitly when
  calling setup)
* from the default value

The code was changed in b4bd6c8 to
allow programmatically and the default value to work correctly but
setting via ansible.cfg/parameter was broken.

This change should fix setting via ansible.cfg and adds unittests for
all three cases

Fixes #23753

(cherry picked from commit d088030)
abadger added a commit that referenced this issue May 1, 2017
@abadger
Facts Timeout was not settable via ansible.cfg
d088030 
The timeout for gathering facts needs to be settable from three places
(highest precedence to lowest):

* programmatically
* ansible.cfg (equivalent to the user specifying it explicitly when
  calling setup)
* from the default value

The code was changed in b4bd6c8 to
allow programmatically and the default value to work correctly but
setting via ansible.cfg/parameter was broken.

This change should fix setting via ansible.cfg and adds unittests for
all three cases

Fixes #23753
@ansibot ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 7, 2018
@oorabona
Copy link

Hello,

I tried with

ansible 2.7.9
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

And /etc/ansible/ansible.cfg as below :

# cat /etc/ansible/ansible.cfg
[defaults]
gather_timeout=20

I still reproduce this error. So it appears that gather_timeout=20 in /etc/ansible/ansible.cfg seems not be taken into account, the 10s timeouts remains and the only working fix is by calling ansible with -a "gather_timeout=20".

Any ideas ?

@ansible ansible locked and limited conversation to collaborators Apr 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
affects_2.3 This issue/PR affects Ansible v2.3 bug This issue/PR relates to a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Facts Timeout was not settable via ansible.cfg abadger/ansible
5 participants
@abadger @oorabona @i1caro @snowsky @ansibot