Package module does not provide with_items optimisation same as yum

[dvershinin]

ISSUE TYPE

• Bug Report
• Feature Idea

COMPONENT NAME

Generic package module

ANSIBLE VERSION

ansible 2.3.0.0                                                                                      │
  config file =                                                                                      │
  configured module search path = Default w/o overrides                                              │
  python version = 2.7.13 (default, Dec 18 2016, 07:03:39) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (cl│
ang-800.0.42.1)]

SUMMARY

According to documentation,

The yum and apt modules use with_items to execute fewer package manager transactions.

One would expect that the generic package module provides the same optimization for with_items. But it seems not to be the case. Not only with_items works slower (if compared to equivalent task using yum module), it also results in failures in some edge cases where order of installing packages without optimization would be important.

STEPS TO REPRODUCE

The following works fine with yum but fails with package:

- name: "Install ClamAV packages"
  package:
    name: "{{ item }}"
  with_items:
    - "clamav"
    - "clamav-data-empty"
    - "clamav-scanner-systemd"
    - "clamav-update"
  become: yes

EXPECTED RESULTS

TASK [Install ClamAV packages] *************************************************
changed: [example.com] => (item=[u'clamav', u'clamav-data-empty', u'clamav-scanner-systemd', u'clamav-update'])

ACTUAL RESULTS

TASK [Install ClamAV packages] *************************************************
changed: [example.com] => (item=clamav)
failed: [example.com] (item=clamav-data-empty) => {"changed": true, "failed": true, "item": "clamav-data-empty", "msg": "Error: clamav-data-empty conflicts with clamav-data-0.99.2-1.el7.noarch\nError: clamav-data conflicts with clamav-data-empty-0.99.2-1.el7.noarch\n", "rc": 1, "results": ["Loaded plugins: fastestmirror\nLoading mirror speeds from cached hostfile\n * base: mirrors.linode.com\n * epel: epel.mirror.angkasa.id\n * extras: mirrors.linode.com\n * remi-php70: mirrors.thzhost.com\n * remi-safe: mirrors.thzhost.com\n * updates: mirrors.linode.com\nResolving Dependencies\n--> Running transaction check\n---> Package clamav-data-empty.noarch 0:0.99.2-1.el7 will be installed\n--> Processing Conflict: clamav-data-0.99.2-1.el7.noarch conflicts data(clamav) < full\n--> Processing Conflict: clamav-data-empty-0.99.2-1.el7.noarch conflicts data(clamav) > empty\n--> Finished Dependency Resolution\n You could try using --skip-broken to work around the problem\n You could try running: rpm -Va --nofiles --nodigest\n"]}
changed: [example.com] => (item=clamav-scanner-systemd)
changed: [example.com] => (item=clamav-update)      

[sivel]

This is the expected behavior. Under the hood the package module is an action plugin that calls the proper underlying package manage module (yum/apt/etc). Unfortunately not all packaging modules support squashing, so we cannot squash the package plugin by default.

To enable squashing of the package plugin, you can define squash_actions in your ansible.cfg file as documented at:

http://docs.ansible.com/ansible/intro_configuration.html#squash-actions

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project

[ssbarnea]

I would vote for enable squashing by default for package.

If one package module is not supporting it is this module bug and normal users should not pay the huge price for that.

[sivel]

We have the intentions on removing squashing altogether from ansible, in favor of just using a list directly.

e.g:

- package:
    name:
      - vim
      - curl
    state: latest

[ssbarnea]

@sivel Can you please add more info about this "idea". What would be the behaviour: installing all of them as a single transaction of with multiple calls to the package manager for each package?

If it would be using multiple steps it could easily translate to huge executions times and annoying lots of people about ansible taking ages to run. The speed differences are not to be ignored at least with yum where with only 10-20 packages, it can easily mean >20mins extra time to run.

Disabling the installation of packages in a single transaction would be a disaster from the performance point of view.

[sivel]

squashing takes the following:

- package:
    name: "{{ item }}"
    state: latest
  with_items:
    - vim
    - curl

And turns it into:

- package:
    name:
      - vim
      - curl
    state: latest

Both of the above are equivalent, and only uses a "single transaction".

Squashing is not portable, and requires a hard coded list of modules that support squashing, we have plans to remove it. There is really no reason to do it, since modules support this functionality natively, by passing a list to name. In the end, the outcome is the same, but the implementation of squashing is no longer needed.