[DEPRECATION WARNING]: DEFAULT_HASH_BEHAVIOUR, proposed alternative not applicable

[karlyan17]

SUMMARY

The DEFAULT_HASH_BEHAVIOUR configuration option will be deprecated in version 2.13. The Proposed alternative to use the combine filter is not a suitable replacement, as a lot of flexibility and severely limiting the feature set of the ansible inventory.

ISSUE TYPE

• Bug Report

COMPONENT NAME

DEFAULT_HASH_BEHAVIOUR deprecation warning

ANSIBLE VERSION

ansible 2.10.3
  config file = /home/user/share/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.8/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.6 (default, Sep 30 2020, 04:00:38) [GCC 10.2.0]

CONFIGURATION

[DEPRECATION WARNING]: DEFAULT_HASH_BEHAVIOUR option, This feature is fragile and not portable, leading to continual confusion and misuse , use the
``combine`` filter explicitly instead. This feature will be removed in version 2.13. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
DEFAULT_HASH_BEHAVIOUR(/home/user/share/ansible/ansible.cfg) = merge
DEFAULT_HOST_LIST(/home/user/share/ansible/ansible.cfg) = ['/home/user/share/ansible/prod']
DEFAULT_JINJA2_EXTENSIONS(/home/user/share/ansible/ansible.cfg) = jinja2.ext.do

OS / ENVIRONMENT

STEPS TO REPRODUCE

A usecase for DEFAULT_HASH_BEHAVIOUR=merge is the merging of firewall rules defined in different groups and hostfile to a single hash of rules, that can then be templated:

$ cat prod/group_vars/all.yaml
iptables:
  all_rule:
    filter:
      INPUT:
        DROP:
          "-I INPUT -m state --state INVALID -j DROP":
          "-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT":
          "-A INPUT -i lo -j ACCEPT":
          "-A INPUT -p icmp -j ACCEPT":
          "-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT":
      FORWARD:
        DROP:
          "-I FORWARD -m state --state INVALID -j DROP":
          "-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT":
      OUTPUT:
        ACCEPT:
          "-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT":

$ cat prod/group_vars/tinc.yaml
iptables:
  tinc_rule:
    filter:
      INPUT:
        DROP:
          "-A INPUT -i eth0 -p tcp -m tcp --dport 655 -j ACCEPT":
          "-A INPUT -i eth0 -p udp -m udp --dport 655 -j ACCEPT":

after merging of the two (or more) hashes the resulting iptables hash is fed to the following template:

$ cat roles/iptables/templates/iptables.j2
{% set full_rules = {} %}
{% for name,rules in iptables.items() %}
{% do full_rules.update((rules|combine(full_rules, recursive=True))) %}
{% endfor %}

{%- for domain,chains in full_rules.items() %}
*{{ domain }}
{% for chain,policies in chains.items() %}
{% for policy,rules in policies.items() %}
:{{ chain }} {{ policy }} [0:0]
{% for rule in rules.keys() %}
{{ rule }}
{% endfor %}
{% endfor %}
{% endfor %}
COMMIT
{% endfor %}

this produces file, that can be read by iptables-restore

This is but one application of such hash merging behaviour. Others are possible.

While it might be possible to construct a role/template/inventory structure using the combine filter to replicate this behaviour, I believe it will be less elegant and adaptable.

Furthermore the combine filter is not a catch-all solution for this dilemma and requires separate refactoring of all application.

EXPECTED RESULTS

Continue to provide the posibility to merge variables on an inventory level.
Alternatively please provide a solution, that can be applied more broadly without the need to refactor all affected roles.

ACTUAL RESULTS

[DEPRECATION WARNING]: DEFAULT_HASH_BEHAVIOUR option, This feature is fragile and not portable, leading to continual confusion and misuse , use the
``combine`` filter explicitly instead. This feature will be removed in version 2.13.

[ansibot]

Files identified in the description:
None

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

We are explicitly removing a 1 stop method of merging data, in favor of requiring authors to make individual explicit decisions for variable merging. This is exactly what was intended.

We do not plan to resurrect such a general variable merging feature.

Duplicate of #72021

[karlyan17]

Unfortunately as @guyhom pointed out in his comment it is not really clear from #72021, what the decision was based on and the meeting notes also have no mention of this planned change.

If the removal of this feature is a fait-a-complit, then it would be helpful to receive at least some more directions, how to solve the problem without refactoring years of code.

thanks

[sivel]

then it would be helpful to receive at least some more directions, how to solve the problem without refactoring years of code.

It will absolutely require refactoring. There is no non-refactoring option.

You may elect to stay on older versions of Ansible that supports the functionality you desire. Ansible 2.9 will be supported until 2023, and Ansible 2.12 may be supported until potentially 2026.