Learn how to run Ansible tasks in parallel. Suitable when calling slow modules, such as cloud modules.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
README.md
run-in-parallel.yml

README.md

Run Ansible tasks in parallel

When you use Ansible to provision hosts, running a task on a set of hosts in parallel is not a problem. You simply use the serial keyword in a playbook. See: http://docs.ansible.com/ansible/latest/playbooks_delegation.html#rolling-update-batch-size

This Ansible repository is about when you want to run a task several times in parallel, either on each host, or on a single host. A common use case is when you want to provision infrastructure at a cloud provider, such as Amazon Web Services (AWS).

Creating a single AWS EC2 instance or a RDS instance takes a long time. If you want to create several, you can use a loop and wait for each to finish, but that takes a long time. As a alternative, you can create them all in parallel.

The playbook run-in-parallel.yml in this repository is an example of how you can run any Ansible task in parallel and wait for them all to finish.

One of the plays runs the tasks in parallel, and one runs then in sequence, for comparison.

How to run example

ansible-playbook run-in-parallel.yml

Example output

$ ansible-playbook run-in-parallel.yml 

PLAY [Run tasks in parallel] **********************************************************************************************************

TASK [Pretend to create instances] ****************************************************************************************************
changed: [localhost] => (item=6)
changed: [localhost] => (item=8)
changed: [localhost] => (item=7)

TASK [Wait for creation to finish] ****************************************************************************************************
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'957883465297.7487', 'item': 6, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/957883465297.7487'})
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'450659566541.7513', 'item': 8, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/450659566541.7513'})
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'763185500456.7538', 'item': 7, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/763185500456.7538'})

PLAY [Run tasks in sequence] **********************************************************************************************************

TASK [Pretend to create instances] ****************************************************************************************************
changed: [localhost] => (item=6)
changed: [localhost] => (item=8)
changed: [localhost] => (item=7)

PLAY RECAP ****************************************************************************************************************************
localhost                  : ok=3    changed=3    unreachable=0    failed=0   

Notes

  • Using Ansible 2.4

Author

Max Wenzin, Crisp