Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wait_for_task backoff behavior in vmware modules #45429

Merged
merged 1 commit into from
Sep 11, 2018

Conversation

albers
Copy link
Contributor

@albers albers commented Sep 10, 2018

SUMMARY

I stepped into this when I tried vmware_guest_powerstate to turn off a running VM in VMware by setting state=powered-off. The instance went down almost immediately, but I had to wait another minute for the module task to return.

In #39596, a timer with an exponential backoff algorithm was added:

sleep_time = min(2 ** failure_counter + randint(1, 1000), max_backoff)

The intent was obviously to wait for [1,2,4,8,16,32,64,64...] seconds, with some fuzzing added to the seconds. But as it is implemented, the added randint(1, 1000) will be greater than max_backoff (64) most of the time, resulting in a 93,6% chance that the first wait duration will be 64 seconds.

I assume that the random value was intended to add a slight fuzzing in the range 0..1000 milliseconds.

This PR adjusts the code to that assumption.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

lib/ansible/module_utils/vmware.py

ANSIBLE VERSION
ansible 2.6.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
ADDITIONAL INFORMATION

sample invocation, credentials specified in environment variables

hacking/test-module -m lib/ansible/modules/cloud/vmware/vmware_guest_powerstate.py -a 'state=powered-off name=some-running-vm'

@ansibot
Copy link
Contributor

ansibot commented Sep 10, 2018

@ansibot ansibot added affects_2.8 This issue/PR affects Ansible v2.8 bug This issue/PR relates to a bug. needs_triage Needs a first human triage before being processed. small_patch support:community This issue/PR relates to code supported by the Ansible community. vmware VMware community labels Sep 10, 2018
@ansibot
Copy link
Contributor

ansibot commented Sep 10, 2018

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/module_utils/vmware.py:61:69: E226 missing whitespace around arithmetic operator

click here for bot help

@ansibot ansibot added ci_verified Changes made in this PR are causing tests to fail. needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Sep 10, 2018
@Akasurde Akasurde self-requested a review September 10, 2018 16:02
@Akasurde Akasurde removed the needs_triage Needs a first human triage before being processed. label Sep 10, 2018
Signed-off-by: Harald Albers <github@albersweb.de>
@ansibot ansibot removed the ci_verified Changes made in this PR are causing tests to fail. label Sep 11, 2018
Copy link
Member

@Akasurde Akasurde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ansibot ansibot removed the needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. label Sep 11, 2018
@Akasurde Akasurde merged commit 796d8b5 into ansible:devel Sep 11, 2018
@albers
Copy link
Contributor Author

albers commented Sep 11, 2018

@Akasurde Thanks for merging so quickly and also for fixing the whitespace issue.

By the way, this fix explains the flakyness reported in #45223: Most of the time, the VMs had 64 seconds time to come up, which would be sufficient for the IP address to become available. A shorter timeout might not be sufficient.

I'm looking forward to some noticable faster executions now.

@albers albers deleted the wait_for_task-backoff branch September 11, 2018 07:45
@ansible ansible locked and limited conversation to collaborators Jul 22, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.8 This issue/PR affects Ansible v2.8 bug This issue/PR relates to a bug. cloud small_patch support:community This issue/PR relates to code supported by the Ansible community. vmware VMware community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants