ansible provisionner: support collections using ansible-galaxy

[nqb]

Hello,

First of all, thanks for this nice software !

Context

Current Ansible provisionner is able to download roles from Ansible Galaxy website during provisioning (option galaxy_role_file).

Since Ansible 2.8 and Galaxy 3.2, users can upload collections on Galaxy website.

Collections could be install using mazer tool and a collections lock file:

$ mazer install --collections-lock collections_lockfile.yml

collections_lockfile.yml (see format):

alikins.collection_inspect: "*"
alikins.collection_ntp: "*"

Request

Have a dedicated option for Ansible provisionner to install collections from a collection lockfile if mazer is present on the host.

[nqb]

According to ansible/ansible#57106, we will not need to support collections lockfile and mazer because support of ansible collections has been added to ansible-galaxy command starting from ansible 2.9.

I will made some tests when Ansible 2.9 will be released.

[gildegoma]

@nqb thank you very much! Please let us know when you have some Ansible 2.9+ results 👍

[nqb]

Hello @gildegoma,

According to following documentations:

• Using collections
• Developing collections

ansible-galaxy collection is now the command to manage collections, mazer has been deprecated.

A requirements file can be used to install collections and roles. Example:

---
roles:
  - src: inverse_inc.gitlab_buildpkg_tools

collections:
  - name: inverse_inc.packetfence

BUT:

While both roles and collections can be specified in one requirements file, they need to be installed separately. The ansible-galaxy role install -r requirements.yml will only install roles and ansible-galaxy collection install -r requirements.yml -p ./ will only install collections.

[nqb]

We can have same usage as for roles with following command: 

ansible-galaxy collection install -r requirements.yml -p collections`

This will install collections in a relative collections directory which will be find by Ansible even if this path is not part of COLLECTIONS_PATHS (see https://docs.ansible.com/ansible/2.9/user_guide/collections_using.html#installing-collections-with-ansible-galaxy)

[nqb]

We certainly need to use a new compatibility mode for Ansible 2.9

[IyadKandalaft]

+1

[nqb]

I did a test today with:

• Vagrant 2.2.10
• Ansible 2.10.2 (not ansible-base)
• following requirements.yml in my VAGRANT_CWD dir:

---
roles:
  - src: geerlingguy.nodejs
  - src: inverse_inc.gitlab_buildpkg_tools

collections:
  - name: inverse_inc.packetfence
  - name: debops.debops
  - name: inverse_inc.windows
  - name: inverse_inc.cumulus
  - name: inverse_inc.utils

• my usual Ansible provisioner:

  config.vm.provision "ansible", type: 'ansible' do |ansible|
    ansible.playbook = "site.yml"
    ansible.config_file = "ansible.cfg"
    ansible.inventory_path = "inventory"
    ansible.galaxy_role_file = "requirements.yml"
end

• following ansible.cfg:

[defaults]
inventory = inventory/

# change display of log
stdout_callback = yaml

# disable host key checking
host_key_checking = False

# forks
forks = 50

# Installs collections into [current dir]/ansible_collections/namespace/collection_name
collections_paths = ./

And due to changes related to ansible-galaxy command between 2.9 and 2.10, Vagrant Ansible provisioner installed roles and collections in my VAGRANT_CWD and everything works !

[elreydetoda]

Trying to do ansible_local local provisioner, this doesn't work for me. So, as a hack I had to add the following to handle the installation of collections

ansible.galaxy_command = "sudo ansible-galaxy collection install -r %{role_file} --force && sudo ansible-galaxy role install -r %{role_file} --force"

which will just install it to the default collections location, but if people wanted to do the path as well ( like mentioned here: #10958 (comment) ) then they should be able to add -p %{roles_path} like mentioned in this tips and tricks section: https://www.vagrantup.com/docs/provisioning/ansible_local#install-galaxy-roles-in-a-path-owned-by-root

[bruinsg]

If you are using the ansible_local local provisioner you need to set ansible.compatibility_mode = "2.0" and then the roles and collections will be both installed without having to specify the galaxy_command

[strarsis]

Is the ansible.cfg used with the remote ansible provider? Ansible itself supports the collections_path config field - shouldn't this be definable using that configuration file?

@elreydetoda: It doesn't work for me sadly, I get an error instead:

ERROR! The positional collection_name arg and --requirements-file are mutually exclusive.

Edit: I solved this now with an external script:
ansible-galaxy-install-all.sh:

#!/usr/bin/env sh
ansible-galaxy collection install --collections-path="$3" --force -r "$2" && \
ansible-galaxy role       install       --roles-path="$1" --force -r "$2"

Vagrantfile

# [...]
ansible.galaxy_command = './scm/ansible-galaxy-install-all.sh %{roles_path} %{role_file} scm/vendor/collections'
# [...]

The placeholders can be passed as arguments (and the path to collections directory) to the external script so they can be also used there.

[elreydetoda]

ya... @strarsis, I think I had this error recently as well. I think this happened in some kind of update, but didn't dig too far into it. I will post back here if I can find the vagrantfile this was happening in, but I do have quite a few of them that I have collected over the years 😅 so I don't know when I will get a chance to post it.

Also, cool @uncletall! I will have to check that out. I have so many different things that I need to add by default now, for Vagrantfiles, I feel like I need to create a script to generate all of the default config options 😅 ( depending on provisioner and provider of course ).

[jamiejackson]

@bruinsg, this didn't seem to work:

If you are using the ansible_local local provisioner you need to set ansible.compatibility_mode = "2.0" and then the roles and collections will be both installed without having to specify the galaxy_command

@elreydetoda, this almost worked for me, but not quite:

ansible.galaxy_command = "sudo ansible-galaxy collection install -r %{role_file} --force && sudo ansible-galaxy role install -r %{role_file} --force"

I just had to remove the sudos so that it installed as the vagrant user, instead of root.

ansible.galaxy_command = "ansible-galaxy collection install -r %{role_file} --force && ansible-galaxy role install -r %{role_file} --force"

Thanks for the workaround!