Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix issues with ansible-playbook-callbacks test #82407

Merged
merged 1 commit into from
Jan 17, 2024
Merged

Fix issues with ansible-playbook-callbacks test #82407

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 7 additions & 7 deletions test/integration/targets/ansible-playbook-callbacks/all-callbacks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,22 +96,22 @@
ignore_errors: true

- name: async poll ok
command: sleep 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is needed as I have yet to see this being an issue in our CI. Generally we want the sleep to be as short as possible to reduce tests running time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely hit the issue locally, and I saw it in CI once too, but that may have been with an earlier iteration of the values. Either way, it should be possible to run the integration tests on a laptop reliably.

I understand the concern around test runtime, and set it up to ensure that only one poll callback is sent per task.

The problem is that the sleep duration is a multiple of the poll interval, so the number of polls can vary. I chose these numbers to make the sleep finish in the middle of the 1st and 2nd poll.

I'll try halving all of the numbers, which should reduce the task duration to 2s, as it was before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm testing this locally like this:

for i in $(seq 1 10); do if ! ansible-test integration ansible-playbook-callbacks; then echo "Failed on $i"; break; fi; done

With the original numbers (sleep 2 poll 1) and it failed on the 2nd attempt. Tried again and it failed on the 5th attempt.

I tried it twice with the change proposed here (sleep 3 poll 2) and it did not fail.

I tried it with the numbers halved (sleep 1.5 poll 1) and it failed on the second attempt.

I tried a few other combinations, but couldn't find something that worked well (poll must be an integer).

So I'm inclined to keep the numbers proposed here - a slow test is better than an unreliable one.

async: 3
poll: 1
command: sleep 3
async: 5
poll: 2

- name: async poll failed
shell: sleep 2; false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

async: 3
poll: 1
shell: sleep 3; false
async: 5
poll: 2
ignore_errors: true

- include_tasks: include_me.yml
markgoddard marked this conversation as resolved.
Show resolved Hide resolved

- name: diff
copy:
content: diff
dest: '{{ remote_tmp_dir.path }}/diff.txt'
dest: '{{ remote_tmp_dir }}/diff.txt'
diff: true

- hosts: i_dont_exist
markgoddard marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
12 changes: 7 additions & 5 deletions test/integration/targets/ansible-playbook-callbacks/callbacks_list.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
1 __init__
83 v2_on_any
92 v2_on_any
1 v2_on_file_diff
4 v2_playbook_on_handler_task_start
2 v2_playbook_on_include
1 v2_playbook_on_no_hosts_matched
3 v2_playbook_on_notify
1 v2_playbook_on_play_start
3 v2_playbook_on_play_start
1 v2_playbook_on_start
1 v2_playbook_on_stats
17 v2_playbook_on_task_start
19 v2_playbook_on_task_start
1 v2_playbook_on_vars_prompt
1 v2_runner_item_on_failed
2 v2_runner_item_on_ok
Expand All @@ -15,8 +17,8 @@
1 v2_runner_on_async_ok
2 v2_runner_on_async_poll
5 v2_runner_on_failed
15 v2_runner_on_ok
16 v2_runner_on_ok
1 v2_runner_on_skipped
21 v2_runner_on_start
23 v2_runner_on_start
1 v2_runner_on_unreachable
2 v2_runner_retry
Empty file added test/integration/targets/ansible-playbook-callbacks/include_me.yml
View file
Open in desktop
Empty file.
1 change: 1 addition & 0 deletions test/integration/targets/ansible-playbook-callbacks/runme.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ set -eux
export ANSIBLE_CALLBACK_PLUGINS=../support-callback_plugins/callback_plugins
export ANSIBLE_ROLES_PATH=../
export ANSIBLE_STDOUT_CALLBACK=callback_debug
export ANSIBLE_HOST_PATTERN_MISMATCH=warning
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think this should be set for the all-callbacks.yml playbook only as opposed to the whole script in case more tests are added in the future but not a blocker.


ansible-playbook all-callbacks.yml 2>/dev/null | sort | uniq -c | tee callbacks_list.out

Expand Down