azure_rm_loadbalancer doesn't append rules, it overwrites

[rnsc]

SUMMARY

I want to update the "load balancing rules" of a basic Azure Load Balancer.
In Ansible 2.8, that module was updated for idempotency and it can be used to updated an existing LB object.
However, it doesn't append the new rules to existing rules in a LB, it just overwrites all the rules.
I was expecting the module to just "append" rules to the current set.

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_loadbalancer

ANSIBLE VERSION

ansible 2.8.1
  config file = [redacted]/ansible.cfg
  configured module search path = [u'[redacted]/library', u'[redacted]/ara/plugins/modules']
  ansible python module location = [redacted]/local/lib/python2.7/site-packages/ansible
  executable location = [redacted]/bin/ansible
  python version = 2.7.15rc1 (default, Nov 12 2018, 14:31:15) [GCC 7.3.0]

CONFIGURATION

ANSIBLE_NOCOWS([redacted]/ansible.cfg) = True
ANSIBLE_PIPELINING([redacted]/ansible.cfg) = True
ANSIBLE_SSH_ARGS([redacted]/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=600s -o StrictHostKeyChecking=no
ANSIBLE_SSH_RETRIES([redacted]/ansible.cfg) = 3
CACHE_PLUGIN([redacted]/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION([redacted]/ansible.cfg) = ~/.ansible/facts.cachedir
CACHE_PLUGIN_TIMEOUT([redacted]/ansible.cfg) = 300
DEFAULT_ACTION_PLUGIN_PATH([redacted]/ansible.cfg) = [u'[redacted]/ara/plugins/actions']
DEFAULT_CALLBACK_PLUGIN_PATH([redacted]/ansible.cfg) = [u'[redacted]/ara/plugins/callbacks']
DEFAULT_CALLBACK_WHITELIST([redacted]/ansible.cfg) = [u'profile_roles', u'profile_tasks', u'timer', u'junit']
DEFAULT_FORKS([redacted]/ansible.cfg) = 100
DEFAULT_GATHERING([redacted]/ansible.cfg) = smart
DEFAULT_HOST_LIST([redacted]/ansible.cfg) = [u'[redacted]/slinventory.sh']
DEFAULT_LOG_PATH([redacted]/ansible.cfg) = /home/renaud/.ansible/SLAnsible.log
DEFAULT_LOOKUP_PLUGIN_PATH([redacted]/ansible.cfg) = [u'[redacted]/plugins/lookup']
DEFAULT_MODULE_PATH([redacted]/ansible.cfg) = [u'[redacted]/library', u'[redacted]/ara/plugins/modules'
DEFAULT_REMOTE_USER([redacted]/ansible.cfg) = stylelabs
DEFAULT_ROLES_PATH([redacted]/ansible.cfg) = [u'[redacted]/roles_galaxy', u'[redacted]/roles_mansible']
DEFAULT_STDOUT_CALLBACK([redacted]/ansible.cfg) = yaml
DEFAULT_STRATEGY([redacted]/ansible.cfg) = mitogen_linear
DEFAULT_STRATEGY_PLUGIN_PATH([redacted]/ansible.cfg) = [u'[redacted]/mitogen/ansible_mitogen/plugins/strategy']
DEFAULT_TIMEOUT([redacted]/ansible.cfg) = 20
HOST_KEY_CHECKING([redacted]/ansible.cfg) = False
RETRY_FILES_ENABLED([redacted]/ansible.cfg) = False

OS / ENVIRONMENT

Running on localhost.

STEPS TO REPRODUCE

Create a basic load balancer on Azure with a Frontend config and BackendPool.
Add a Load balancing rule manually with a health probe on port 80.
Execute Ansible task to add a new rule.

- name: "azure_rm_loadbalancer | add LB rule and probe"
  azure_rm_loadbalancer:
    auth_source: cli
    subscription_id: "REDACTED"
    name: "elbtest"
    resource_group: "elb-test"
    load_balancing_rules:
      - backend_address_pool: "backend"
        backend_port: "1024"
        frontend_ip_configuration: LoadBalancerFrontEnd
        frontend_port: "1024"
        name: "1024-rule"
        probe: "1024-probe"
        protocol: Tcp
    probes:
      - name: "1024-probe"
        port: "1024"
        protocol: Tcp

EXPECTED RESULTS

I'm expecting just my rule to be added, unless I specifically say I want to purge everything.

ACTUAL RESULTS

The rules and probes that are not part of the current Ansible task are deleted.

[ansibot]

Files identified in the description:

• lib/ansible/modules/cloud/azure/azure_rm_loadbalancer.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @haroldwongms @nitzmahone @trstringer @yungezz @yuwzho @zikalino
click here for bot help

[ansibot]

@rnsc, just so you are aware we have a dedicated Working Group for azure.
You can find other people interested in this in #ansible-azure on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

[mmyyrroonn]

@rnsc Hi. I'm not sure it's a bug or by designed. In my understanding, the playbook file is a description for the status. So the overwrite behavior makes sense for such list field.

[rnsc]

@myronfanqiu Hello, thanks for your reply.
If it's by design, then, it should be stated as is in the documentation, otherwise it's confusing. I was expecting this to be able to just add values for the "load balancing rules", "probes" and "inbound nat rules".

Do we want to change this to a feature request then? Asking for an update/overwrite mechanic in the module?

If I take the "azure_rm_securitygroup" module as an example: https://docs.ansible.com/ansible/latest/modules/azure_rm_securitygroup_module.html#azure-rm-securitygroup-module
There we have a "purge_rules" flag that can be used to specifically say that we want to blank everything and put the new rules in place.

[mmyyrroonn]

@rnsc Hi. Thanks for this suggestion. Of course, we can change this to a feature request. Honestly, the update behavior for such list filed across all azure modules are not totally consistent. From my perspective, there are two reasons for such situation. First, the behavior of each azure service are not same. Some rest API will do the incremental update and some will do the replace update for such list field. Thus, there isn't a good guild line for implementing such field in ansible. The other reason is the author of different modules are different. 😃
Anyway, we can change this to a feature request and we will track this one. It may need some time to discuss. We will share any update with you. Thanks again!

[mmyyrroonn]

@zikalino @yungezz FYI. Please look at this feature request when you're available.

[mw-root]

I entirely understand the frustration with Azure REST APIs.. this is why I use ansible ;)

I also believe there at least needs to be a flag to set to append/overwrite. Many of us are adding LB rules as products and services come online. We need to be able to add these rules without going out and pulling all existing rules, frontend ip configs, and probes and adding them to the task.

[Fred-sun]

@yungezz @zikalino

[Fred-sun]

@yungezz @zikalino New feature request, please take a look when you're free! Thank you very much!

[Fred-sun]

@yungezz @zikalino

[Fred-sun]

@mybayern1974 Please take a look this issue when you're free! Thank you very much!

[Fred-sun]

@mybayern1974 @haiyuazhang Please help!

[ansibot]

cc @gavinfish
click here for bot help

[Fred-sun]

@haiyuazhang @gavinfish Please look at this feature request when you're available. Thank you very much!

[Fred-sun]

Working in!

[Fred-sun]

@rnsc Thank you very much for your interest in Ansible. This module is no longer maintained in this repository and has been migrated. Since this change is specific to AZURE module and not for entire repo. Please re-submit this issue in the https://github.com/ansible-collections/azure and closed this. Thank you very much!

[rnsc]

@Fred-sun thanks, created: ansible-collections/azure#112