The value of 'serial' in playbook cannot be set based on a variable defined per group

[FlorinAndrei]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

playbook

ANSIBLE VERSION

ansible 1.9.4
  configured module search path = None

CONFIGURATION

OS / ENVIRONMENT

N/A

SUMMARY

Doing rolling updates with Ansible.

ansible-playbook -i production deploy.yml

ansible-playbook -i staging deploy.yml

$ cat deploy.yml

---
- hosts: api_frontend
  serial: 4

  pre_tasks:
    - name: "Gather EC2 facts from API nodes"
      ec2_facts:

    - name: "Remove node from ELB"
      local_action: ec2_elb
      args:
        instance_id: "{{ ansible_ec2_instance_id }}"
        region: "{{ aws.region }}"
        ec2_elbs: "{{ aws_api_lb }}"
        state: "absent"

  roles:
    - { role: api-deploy }

  post_tasks:
    - name: "Add node to ELB"
      local_action: ec2_elb
      args:
        instance_id: "{{ ansible_ec2_instance_id }}"
        region: "{{ aws.region }}"
        ec2_elbs: "{{ aws_api_lb }}"
        state: "present"
        wait_timeout: 30

The problem is, serial: 4 works well in production, but the value is too large for staging. In staging I'd like to use serial: 1.

I've tried to define a variable such as api_deploy_serial in places like production/group_vars/all/vars (and also for staging), and used that to define the value for serial in the playbook, but it does not seem to be taken into account. Seems like variables are evaluated after serial is used in the playbook.

I do not want to maintain multiple deploy playbooks, one for each environment. That playbook needs to be created once, and pull variables from various environments for various behaviors. That's how config management is supposed to work.

Having multiple playbooks, one per environment, without the ability to template the behavior in the standard way, would be like a throw back to the bad old days of shell scripts.

STEPS TO REPRODUCE

If I do:

serial: "{{ api_deploy_serial | default(1) | int }}"

then only the default value is used, no matter what I add to group_vars.

EXPECTED RESULTS

ACTUAL RESULTS

[krzysztof-magosa]

What would happen if two groups have conflicting value of variable for serial?

[FlorinAndrei]

What happens when two groups have conflicting values of any variable?

[bcoca]

The groups are not an issue, that each host has a different value is, as vars are flattened by host so groups have no bearing after that.

[krzysztof-magosa]

@FlorinAndrei Normally each host would have some value calculated based on precedence, but I believe value of serial must be 'global' for entire play and cannot be different for hosts within it.

For example something like that:

- hosts:
    - frontend
    - backend
  serial: "{{ some_variable }}"
  roles:
    - role1
    - role2
    - role3

And assume that frontend and backend group does not share the same hosts, and some_variable is set to different value for each group.

[FlorinAndrei]

I just want some way to template that variable, the way many other variables can be templated. It's a bit crazy to have to use different versions of the same playbook for different environments, only because one variable is hardcoded. I would like to be able to pull its value from somewhere, no matter how many times it could potentially be overridden.

I'm sure there are good structural reasons for the current behavior, but from the user's perspective Ansible seems a little inconsistent. Some variables can be pulled from groups. Others can't. It's difficult to know what the logic is until you actually try to see if it works. :(

[bcoca]

you can template it, you just cannot use host variables to do so

[pabelanger]

Just ran into this issue too, was trying to have Zuul dynamically do this for me via host vars. Back to the drawing board.

[danowar2k]

I've just run into this problem. My Ansible CM is a VM with X MB RAM. Each host consumes ca. 300 MB for the tasks I'm running. So I first set serial hard to 3.

But what if I decide to give more or less RAM to my Ansible CM? So I started to create some tasks to dynamically set the amount of servers this play can do in parallel:

- hosts: all
  name: Store CM dependent variables
  tasks:
    - name: Check Ansible CM memory
      local_action:
        module: setup
      register: ansible_cm_facts

    - name: Set server worker count depending on Ansible CM memory
      set_fact:
        parallel_server_workers: "{{ ansible_cm_facts.ansible_facts.ansible_memtotal_mb / 300) | int }}"

- hosts: all
  name: Memory intensive play
  serial: "{{ parallel_server_bla }}"

Which didn't work because I can't use host variables and Control Machine variables aren't a thing that exists. So, it would be nice if I could do this at some time in the future...

[bcoca]

@danowar2k 'control machine variables' would be 'extra vars' -e myserial=32 or play vars i.e vars_prompt/vars_files

[danowar2k]

How would I detect how much memory the Ansible CM has and set some of those variables with it?

Do you have an example?

[bcoca]

a 'pipe' lookup could get you that info

[danowar2k]

I'll try that out.

Meanwhile, a warning to those who try my method above. When I do the setup module as a local action, Ansible overwrites the setup facts for every host with facts about the Ansible CM. I thought I could avoid that with register, but it didn't work. Maybe a switch for setup could prevent this...

[bcoca]

@danowar2k your local_action would work if you add delegate_facts: true then it would apply those to localhost instead on each host

[danowar2k]

Thanks, I didn't yet know about that.

[bcoca]

Since this is a hard limitation on what a play is and how hosts work, I'm going to close this issue as we don't see this feature being implemented.