Blocks: rescue or always not run when error comes from a run_once task #16937

Gugli · 2016-08-03T15:29:48Z

ISSUE TYPE

Bug Report

ANSIBLE VERSION

ansible 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

Out of the box config on Ubuntu14

OS / ENVIRONMENT

Ansible runs on Ubuntu14, targeted machines on Ubuntu16.

SUMMARY

Using run_once on a task inside a block prevents rescue and always sections to be correctly triggered.

STEPS TO REPRODUCE

Run this playbook on any inventory file. This is basically the sample from the official documentation with just a run_once added

# Contents of bug_blocks.yml
- hosts: all
  gather_facts: false
  tasks:
  - block:
      - debug: msg='I execute normally'
      - command: /bin/false
        run_once: true
      - debug: msg='I never execute, cause ERROR!'
    rescue:
      - debug: msg='I caught an error'
      - command: /bin/false
      - debug: msg='I also never execute :-('
    always:
      - debug: msg="this always executes"

EXPECTED RESULTS

I was expecting

$ ansible-playbook bug_blocks.yml --inventory-file inventories/

PLAY [all] *********************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "i execute normally"
}

TASK [command] *****************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/bin/false"], "delta": "0:00:00.002056", "end": "2016-08-03 17:04:07.156094", "failed": true, "rc": 1, "start": "2016-08-03 17:04:07.154038", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "I caught an error"
}

TASK [command] *****************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/bin/false"], "delta": "0:00:00.001853", "end": "2016-08-03 17:04:07.988953", "failed": true, "rc": 1, "start": "2016-08-03 17:04:07.987100", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "this always executes"
}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=1
TASK: command ----------------------------------------------------------- 0.79s
TASK: command ----------------------------------------------------------- 0.40s
TASK: debug ------------------------------------------------------------- 0.11s
TASK: debug ------------------------------------------------------------- 0.09s
TASK: debug ------------------------------------------------------------- 0.07s

Playbook finished: Wed Aug  3 17:04:08 2016, 5 total tasks.  0:00:01 elapsed.

ACTUAL RESULTS

$ ansible-playbook bug_blocks.yml --inventory-file inventories/

PLAY [all] *********************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "i execute normally"
}

TASK [command] *****************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/bin/false"], "delta": "0:00:00.001284", "end": "2016-08-03 17:03:52.951110", "failed": true, "rc": 1, "start": "2016-08-03 17:03:52.949826", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0

TASK: command ----------------------------------------------------------- 0.71s
TASK: debug ------------------------------------------------------------- 0.09s

Playbook finished: Wed Aug  3 17:26:37 2016, 2 total tasks.  0:00:00 elapsed.

Related issues

This might be related to #16254, but the steps to reproduce are so different I though it best to make a new Issue report.

The text was updated successfully, but these errors were encountered:

Gugli · 2016-08-03T15:32:49Z

Might also be somewhat related to #16050.

MadVikingGod · 2016-08-04T18:56:17Z

I can confirm this bug. It does not happen on v2.1.0.0-1, but everything later does not work correctly.

I have narrowed it down to fbec2d9 linear.py lines 364 and 365.

Reverting this part of that patch will fix this but, but I think will also undo the fix for #16937.

A real fix for both of these issues is to not treat run_once as any_error_fatal, and have a different logic path if it fails. This would mark all of the hosts in that play as failed, and then run the appropriate recovery logic.

jctanner · 2016-08-05T17:12:10Z

cc @jimi-c

Instead of immediately returning a failed code (indicating a break in the play execution), we internally 'or' that failure code with the result (now an integer flag instead of a boolean) so that we can properly handle the rescue/always portions of blocks and still remember that the break condition was hit. Fixes #16937

jimi-c · 2016-08-05T18:55:09Z

@Gugli / @MadVikingGod the above branch is a first attempt at fixing this. Still needs some work but I think the basics are there. I still need to:

go through our other strategy plugins and base strategy to make sure at least in our code we're using integer results values now
my logic for the handler stuff (and vs. or) is probably not right, but haven't tested it yet (shippable is running on it)
only the first host in the list executes the rescue - no clue as to why yet because the rest are marked as failed and the PlayIterator should have them all at the same point too.

jimi-c · 2016-08-05T18:57:01Z

Hrm, and it looks like I've got some other bit of logic wrong.. this test succeeded but there was a non-zero exit code:

https://app.shippable.com/runs/57a4dfa5d640fc0d00f90f97/5/console

jimi-c · 2016-08-05T18:57:33Z

$ ansible -m ping localhost
localhost | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
$ echo $?
1

In the linear strategy the flag run_once was synomious with any_error_fatal for error checking. This caused plays to abort before running any rescue or always blocks. By remove this the play executes as expected, and the error handeling is handled by the executor. This is a fix for ansible#16937

Instead of immediately returning a failed code (indicating a break in the play execution), we internally 'or' that failure code with the result (now an integer flag instead of a boolean) so that we can properly handle the rescue/always portions of blocks and still remember that the break condition was hit. Fixes #16937

pporada-gl · 2016-08-12T13:40:42Z

I hit this today on ansible 2.1.1.0 without the run_once.

Instead of immediately returning a failed code (indicating a break in the play execution), we internally 'or' that failure code with the result (now an integer flag instead of a boolean) so that we can properly handle the rescue/always portions of blocks and still remember that the break condition was hit. Fixes #16937

jimi-c · 2016-08-12T17:21:42Z

The branch above is working and passing tests, if anyone else would like to give it a test before I merge it into devel.

jimi-c · 2016-08-12T19:50:27Z

Closing This Ticket

Hi!

We believe the above commit should resolve this problem for you. This will also be included in the next release.

If you continue seeing any problems related to this issue, or if you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:

https://groups.google.com/forum/#!forum/ansible-project - for user questions, tips, and tricks
https://groups.google.com/forum/#!forum/ansible-devel - for strategy, future planning, and questions about writing code

Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular issue is resolved.

Thank you!

jimi-c · 2016-08-15T19:44:55Z

Closing This Ticket

Hi!

We believe the above commit should resolve this problem for you. This will also be included in the next release.

If you continue seeing any problems related to this issue, or if you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:

https://groups.google.com/forum/#!forum/ansible-project - for user questions, tips, and tricks
https://groups.google.com/forum/#!forum/ansible-devel - for strategy, future planning, and questions about writing code

Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular issue is resolved.

Thank you!

jctanner added the bug_report label Aug 5, 2016

jctanner assigned jimi-c Aug 5, 2016

jimi-c added this to the stable-2.1 milestone Aug 5, 2016

MadVikingGod mentioned this issue Aug 8, 2016

Removed run_once from alias of any error fatal #17005

Closed

jimi-c closed this as completed in c669a38 Aug 12, 2016

jimi-c mentioned this issue Oct 14, 2016

any_errors_fatal doesn't work any more #18021

Closed

jctanner mentioned this issue Mar 13, 2017

Handlers do not get executed if 'run_once: True' and execution fails (even with force_handlers: True) #22408

Closed

ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 7, 2018

ansible locked and limited conversation to collaborators Apr 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blocks: rescue or always not run when error comes from a run_once task #16937

Blocks: rescue or always not run when error comes from a run_once task #16937

Gugli commented Aug 3, 2016

Gugli commented Aug 3, 2016

MadVikingGod commented Aug 4, 2016

jctanner commented Aug 5, 2016

jimi-c commented Aug 5, 2016

jimi-c commented Aug 5, 2016

jimi-c commented Aug 5, 2016

pporada-gl commented Aug 12, 2016 •

edited

jimi-c commented Aug 12, 2016

jimi-c commented Aug 12, 2016

jimi-c commented Aug 15, 2016

Blocks: rescue or always not run when error comes from a run_once task #16937

Blocks: rescue or always not run when error comes from a run_once task #16937

Comments

Gugli commented Aug 3, 2016

ISSUE TYPE

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

Related issues

Gugli commented Aug 3, 2016

MadVikingGod commented Aug 4, 2016

jctanner commented Aug 5, 2016

jimi-c commented Aug 5, 2016

jimi-c commented Aug 5, 2016

jimi-c commented Aug 5, 2016

pporada-gl commented Aug 12, 2016 • edited

jimi-c commented Aug 12, 2016

jimi-c commented Aug 12, 2016

Closing This Ticket

jimi-c commented Aug 15, 2016

Closing This Ticket

pporada-gl commented Aug 12, 2016 •

edited