Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: looping over blocks #13262

Closed
smiller171 opened this issue Nov 23, 2015 · 154 comments
Closed

feature request: looping over blocks #13262

smiller171 opened this issue Nov 23, 2015 · 154 comments

Comments

@smiller171
Copy link
Contributor

@smiller171 smiller171 commented Nov 23, 2015

Issue Type:

Feature Idea

Component Name:

blocks

Ansible Version:

Ansible 2.0.0_rc-1

Ansible Configuration:

NA

Environment:

Ubuntu 15.10

Summary of Decision:

We're open to implementing this but want it to go through the proposal process. Please see: #13262 (comment) for details.

Summary:

There are a number of use-cases where it would be valuable to be able to loop over a block of tasks, such that a few tasks are done in order, and that specific block of tasks are looped over for some set of values. It seems that the new block functionality could lend itself well to this if you were to enable looping over blocks.

Steps To Reproduce:
- hosts: localhost
  connection: local
  tasks:
  - block:
    - debug: msg="task 1 loop {{item}}"
    - debug: msg="task 2 loop {{item}}"
    with_items:
    - "1"
    - "2"
Expected Results:
PLAY ***************************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [debug msg=task 1 loop {{item}}] ******************************************
ok: [localhost] => {
    "changed": false, 
    "msg": "task 1 loop 1"
}

TASK [debug msg=task 2 loop {{item}}] ******************************************
ok: [localhost] => {
    "changed": false, 
    "msg": "task 2 loop 1"
}

TASK [debug msg=task 1 loop {{item}}] ******************************************
ok: [localhost] => {
    "changed": false, 
    "msg": "task 1 loop 2"
}

TASK [debug msg=task 2 loop {{item}}] ******************************************
ok: [localhost] => {
    "changed": false, 
    "msg": "task 2 loop 2"
}

PLAY RECAP *********************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0
Actual Results:
ERROR! 'with_items' is not a valid attribute for a Block

The error appears to have been in '/root/test.yml': line 5, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  tasks:
  - block:
    ^ here
@bcoca bcoca closed this Nov 23, 2015
@smiller171
Copy link
Contributor Author

@smiller171 smiller171 commented Nov 23, 2015

@bcoca no, I absolutely understand the functionality here. Currently you can only loop over a certain task, but I want to be able to loop over a set of tasks in series. I need to run (task 1 then task 2)x3 instead of (task 1)x3 then (task2)x3

Currently the only way to do this is to create a script that does tasks 1 and 2, or perhaps call a playbook from within a playbook.

Looping over roles would allow for the same behavior.

@bcoca
Copy link
Member

@bcoca bcoca commented Nov 23, 2015

you can loop over includes, sorry about the response, I misread this as a bug report

@bcoca bcoca reopened this Nov 23, 2015
@smiller171
Copy link
Contributor Author

@smiller171 smiller171 commented Nov 23, 2015

@bcoca I hadn't seen the docs on looping over includes, which seems the better solution than what I have suggested, as my method could quickly result in code that is more difficult to follow.

@messiahUA
Copy link

@messiahUA messiahUA commented Dec 10, 2015

I want this, because includes are very slow (even after recent fix) and clutter output with full hosts lists and are processed even if skipped (tags).

@RuBiCK

This comment was marked as off-topic.

@hewo

This comment was marked as off-topic.

3 similar comments
@srgvg

This comment was marked as off-topic.

@sergevs

This comment was marked as off-topic.

@Lowess

This comment was marked as off-topic.

@lystor

This comment was marked as off-topic.

@aturetta

This comment was marked as off-topic.

@tyl0r

This comment was marked as off-topic.

@arthur-c

This comment was marked as off-topic.

@piotrminkina

This comment was marked as off-topic.

@adul3

This comment was marked as off-topic.

@apenney

This comment was marked as off-topic.

@nikhilo

This comment was marked as off-topic.

@arthurtsang

This comment was marked as off-topic.

@ssbarnea

This comment was marked as off-topic.

@dejlek

This comment was marked as off-topic.

@steveholden

This comment was marked as off-topic.

@soar

This comment was marked as off-topic.

@ffinfo

This comment was marked as off-topic.

@k15r

This comment was marked as off-topic.

@plombardi89

This comment was marked as off-topic.

2 similar comments
@kevensen

This comment was marked as off-topic.

@GeorgeZhai

This comment was marked as off-topic.

@calfonso calfonso closed this Oct 9, 2017
@smiller171
Copy link
Contributor Author

@smiller171 smiller171 commented Oct 9, 2017

@calfonso looping over includes forces you to break out of the current logic and context-switch when writing or updating playbooks, which isn't always ideal. We are aware of the ability to loop overs includes, but would prefer to be able to loop over blocks if we choose. Closing your most popular (by far) feature request as wontfix is pretty disappointing.

@calston

This comment was marked as off-topic.

@smiller171

This comment was marked as off-topic.

@calfonso

This comment has been hidden.

@smiller171

This comment has been hidden.

@abadger
Copy link
Member

@abadger abadger commented Oct 10, 2017

@calfonso yesterday we decided that this was not a feature that we wanted inside of Ansible. So according to our discussion yesterday, closing this ticket is proper. I think since there's some flip-flopping we desperately need to have a discussion today about the pros and cons of accepting this feature and once we have them there, lay all of the pros and cons out in this ticket, not just the "final" decision as to whether this is a feature we want if someone is willing to do the work or if it is counter to what we want Ansible to do.

People will be disappointed if we decide that the feature is not something that we will put in but they will be able to at least understand why we think the alternative is sufficient and what the cost of putting the new feature in is. And if we decide that we would take this as a PR then future committers will be able to refer to the ratoinale here as to why they should take the feature instead of reject it as well.

@smiller171
Copy link
Contributor Author

@smiller171 smiller171 commented Oct 10, 2017

@abadger If you do choose to close this permanently I think that rationale should be explained in-depth attached to my original comment so that it's easy to access for anyone finding this issue later.

@abadger
Copy link
Member

@abadger abadger commented Oct 11, 2017

We talked about this extensively these last few days and came to the conclusion that we are open to accepting this feature if someone in the community would care to implement it. However, we're going to leave this ticket closed as this is a complex enough problem that we'd like it to go through the Proposal Process instead. To make this into a proposal, the person who wants to implement it should open an issue on https://github.com/ansible/proposals/issues (there's a template to fill out, you can look at past proposals for guidance), put it on the IRC meeting agenda ( https://github.com/ansible/community/issues?q=is%3Aopen%20label%3Ameeting_agenda%20label%3Acore ) and then plan on attending the IRC meetings to discuss how to implement it and get reviews of finished (or half-finished) code.

Do note, there are many pitfalls in implementing this. To get possible implementers started in the right direction, here are some problems and hints that we've identified in our discussions since we started considering this feature:

  • Looping over includes already serves the same function as looping over blocks. The syntax is different but the functionality is the same.
  • A naive approach to looping over blocks (by turning them into dynamic includes behind the scenes) would be possible but it would bring with it all the assumptions that plague dynamic includes without being obvious to the user why a block acted in a different way when it was inside of a loop versus outside.
  • Similarly, statically expanding the loop would entail similar shortcomings found when doing static includes (early evaluation of variables, no inventory variables, etc.).
  • So a non-naive way would have to be sought out for looping over blocks. This probably entails a reworking of core architecture as neither looping over dynamic nor looping over static includes offers a model that we can apply here. Any implementation needs to make sure that blocks inside and outside of loops can:
    • Reference the same set of variables with the same results
    • Needs to consider rescue/always semantics ...are they also looped?
    • Any_errors_fatal, serial, free vs linear strategies and other interactions.

One of the difficulties in implementing features like this is that we are trying to walk a fine line between automation and not becoming a programming language. We want people to be able to declare what their machines look like, using little bits of programming as shortcuts, to make the declarations clearer, and as an escape hatch when declarative forms are not sufficient. We do not want to create an environment where playbooks are mostly imperative code which the next sysadmin has to puzzle through and interpret. Even within the core team we go back and forth on this kind of feature, struggling to balance the needs of functionality, code maintenance, playbook maintenance, overlap with existing features, etc.

Thank you for understanding and thanks to whoever eventually wants to take a stab at implementing this!

@abadger abadger closed this Oct 11, 2017
@abadger

This comment has been hidden.

openstack-gerrit pushed a commit to openstack/openstack-ansible-lxc_container_create that referenced this issue Nov 2, 2017
Kevin Carter
The block/rescue we were using in the mac generation task did not work
as expected. Because we use an iterator on the task and we can't iterate
over the entire block the task would fail when mac address lookups
within a running container didn't already exist but needed to be
created. This resulted in the task failing for a single host and being
removed from the inventory instead of running a rescue for only the
missing network.

The use of block/rescue has been removed. If the feature to loop over a
block is ever implemented [0] we can revisit how this action is done.

[0] - ansible/ansible#13262
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

Change-Id: Ie4bc3b130874047a5cbd36b98ed86a731ae5c317
@berney

This comment was marked as off-topic.

@smiller171

This comment has been hidden.

@staylorx

This comment was marked as off-topic.

@berney

This comment was marked as off-topic.

@smiller171

This comment was marked as off-topic.

@mrkvost

This comment was marked as off-topic.

@smiller171

This comment was marked as off-topic.

@BenJaziaSadok

This comment was marked as off-topic.

@smiller171
Copy link
Contributor Author

@smiller171 smiller171 commented May 23, 2018

@BenJaziaSadok No. see: #13262 (comment) for details.

Looping over includes is the best way to handle this for now.

@ansible ansible locked as resolved and limited conversation to collaborators May 23, 2018
@dagwieers
Copy link
Member

@dagwieers dagwieers commented Sep 28, 2018

I cleaned up this thread to keep the most relevant information visible. At least it makes an easier read for anyone who wants to understand the whats and whys related to this subject. I am tempted to open this issue, on the basis that if anyone wants to implement this, they can. But given the sheer volume of people, I fear this may get out of hand again.

There's also a new feature request #46203 for supporting until-loops on includes (which are currently not supported) and has some rationale why loops on blocks are unlikely to be featured anytime soon.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet