Give control whether loop_control label is private

[andyfeller]

SUMMARY

Enhance user control whether a given item label should be private.

ISSUE TYPE

• Feature Idea

COMPONENT NAME

lib/ansible/plugins/callback/init.py

ADDITIONAL INFORMATION

During #12214, item labels were censored respective of no_log because there were some cases where the labels were private. In cases where the list is a collection of vaulted variables, this would make sense, however this causes confusion and lack of audibility where loop_control label has been explicitly set to a value that is public.

- hosts: localhost
  connection: local
  vars:
    things:
      - label: this is not
        secret: this is secret
      - label: this is not
        secret: this is secret
  tasks:
    - name: "Loop control label shouldn't always be obscured"
      debug:
        msg: "Let me tell you my {{ item.secret }}"
      with_items: "{{ things }}"
      no_log: yes
      loop_control:
        label: "{{ item.label }}"

results in output proves difficult to understand and verify what is going on

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] ***************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Loop control label shouldn't always be obscured] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=None)
ok: [localhost] => (item=None)
ok: [localhost]

PLAY RECAP *********************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

As discussed in #62091, a mechanism similar to playbook prompts private attribute would allow end users informed consent to output item labels.

[ansibot]

Files identified in the description:

• lib/ansible/plugins/callback/__init__.py
• lib/ansible/plugins/callback/actionable.py
• lib/ansible/plugins/callback/aws_resource_actions.py
• lib/ansible/plugins/callback/cgroup_memory_recap.py
• lib/ansible/plugins/callback/cgroup_perf_recap.py
• lib/ansible/plugins/callback/context_demo.py
• lib/ansible/plugins/callback/counter_enabled.py
• lib/ansible/plugins/callback/debug.py
• lib/ansible/plugins/callback/default.py
• lib/ansible/plugins/callback/dense.py
• lib/ansible/plugins/callback/foreman.py
• lib/ansible/plugins/callback/full_skip.py
• lib/ansible/plugins/callback/grafana_annotations.py
• lib/ansible/plugins/callback/hipchat.py
• lib/ansible/plugins/callback/jabber.py
• lib/ansible/plugins/callback/json.py
• lib/ansible/plugins/callback/junit.py
• lib/ansible/plugins/callback/log_plays.py
• lib/ansible/plugins/callback/logdna.py
• lib/ansible/plugins/callback/logentries.py
• lib/ansible/plugins/callback/logstash.py
• lib/ansible/plugins/callback/mail.py
• lib/ansible/plugins/callback/minimal.py
• lib/ansible/plugins/callback/nrdp.py
• lib/ansible/plugins/callback/null.py
• lib/ansible/plugins/callback/oneline.py
• lib/ansible/plugins/callback/osx_say.py
• lib/ansible/plugins/callback/profile_roles.py
• lib/ansible/plugins/callback/profile_tasks.py
• lib/ansible/plugins/callback/say.py
• lib/ansible/plugins/callback/selective.py
• lib/ansible/plugins/callback/skippy.py
• lib/ansible/plugins/callback/slack.py
• lib/ansible/plugins/callback/splunk.py
• lib/ansible/plugins/callback/stderr.py
• lib/ansible/plugins/callback/sumologic.py
• lib/ansible/plugins/callback/syslog_json.py
• lib/ansible/plugins/callback/timer.py
• lib/ansible/plugins/callback/tree.py
• lib/ansible/plugins/callback/unixy.py
• lib/ansible/plugins/callback/yaml.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @Akasurde @akatch @ares @chris-short @dagwieers @ehelms @ekohl @kyleabenson @martinm82 @rrey @ryancurrah @xprazak2 @ysn2233
click here for bot help

[andyfeller]

Here is an excerpt from an actual playbook used to manage OpenShift projects and resources where I want item label to be the resource type and name of what is affected.

TASK [openshift-project-setup : Process additional template] *********************************************************************************************************************************************************************************
ok: [ci] => (item=resources/secret-buildconfig-source.yml.j2)
ok: [ci] => (item=resources/secret-image-pull.yml.j2)
ok: [ci] => (item=resources/secret-image-pull-legacy.yml.j2)
ok: [ci] => (item=secret-mot-mot-env-vars.yml)

TASK [openshift-project-setup : Create resources] ********************************************************************************************************************************************************************************************
ok: [ci] => (item=project/mot-ci)
changed: [ci] => (item=None)
changed: [ci] => (item=None)
ok: [ci] => (item=None)
ok: [ci] => (item=None)
changed: [ci] => (item=rolebinding/admin)
changed: [ci] => (item=rolebinding/edit)
changed: [ci]

In the underlying playbook, I've chose to selectively no_log based on the resource type:

  no_log: "{{ (ops_no_log | bool) and (item.kind | lower == 'secret') }}"
  with_items: "{{ ops_resources | ops_resource_sort_kind_name }}"
  loop_control:
    label: "{{ (item.kind + '/' + item.metadata.name) | lower }}"

[ansibot]

cc @seuf
click here for bot help

[Vladimir-csp]

I can confirm that private items in loops are very hard to identify. Something like this would really help:

loop_control:
  private_label: False

Also, some kind of custom public templated string would be handy for non-loop tasks too.

[ansibot]

Files identified in the description:

• lib/ansible/plugins/callback/__init__.py
• lib/ansible/plugins/callback/default.py
• lib/ansible/plugins/callback/junit.py
• lib/ansible/plugins/callback/minimal.py
• lib/ansible/plugins/callback/oneline.py
• lib/ansible/plugins/callback/tree.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sgreiner]

Would also be useful for the mysql_user module which is warning if we don't specify no_log: yes.
In the output we don't see what is going on now.

Example:

This

- name: mysqlusers | create mysql user accounts
  no_log: yes
  mysql_user:
    login_unix_socket: /x/y/z/mysqld.sock
    name: "{{ item.user.name }}"
    host: "{{ item.host }}"
    password: "{{ item.user.password_hash }}"
    encrypted: yes
    update_password: always
    priv: "{{ item.user.privileges }}"
    state: present
  loop: "{{ mysql[inventory].hosts_users|company_custom_mysql_host_user_list_filter }}"
  loop_control:
    label: "'{{ item.user.name }}'@'{{ item.host }}'"

gives this "great" output:

TASK [mysql : mysqlusers | create mysql user accounts] ********
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
...
...
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN] => (item=None)
ok: [SERVER.FQDN]

[ansibot]

Files identified in the description:

• lib/ansible/plugins/callback/__init__.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[evgeni]

bot_broken

there is nothing foreman related in this issue, and yet is has the foreman label

[mkrizek]

!bot_broken

[quiescer]

Making this proposed feature 'opt-in' doesn't seem to have any downsides to me. When items in a loop fail, not being able to see which ones failed is less than ideal.

[phemmer]

I see many comments that the reason for this is because loop labels may contain private data. However I can't find any mention of whether these comments are talking about the default label or an explicit label. I can understand why the default label (what is used when no loop_control: { label: ... } is present) should not be shown when no_log: true. However if the user has explicitly defined a label, I cant see why that label should be hidden. Doing so makes defining the explicit label completely pointless.

So why not pursue this as the solution? It would be a lot more intuitive if explicit labels were shown when using no_log: true.

If there is some real world case where a loop label is explicitly defined, and it should still be hidden, can someone provide a concrete example? I couldn't find one digging through the linked issues.

[jhg03a]

Suppose you have a list of dictionaries you're looping over and one of the attributes of said dictionary is sensitive. Also suppose that the task itself emits that sensitive element unless no_log: true is specified. Today, you have no idea where in processing of that list you are if/when it fails. By permitting the use of both no_log: true and loop_control: { label: ... } it's possible to know where things failed and how far you are during execution.

[phemmer]

@jhg03a Was that supposed to be a response to my "concrete example" request? If so that's not quite what I was asking. That's the behavior I was proposing. I was trying to see if there was a real world example where that wouldn't work.

[jhg03a]

Ah, my mistake. I can't think of a functional reason it doesn't.

[ansibot]

Thank you very much for your submission to Ansible. It means a lot to us that you've taken time to contribute.

Unfortunately, this issue has been open for some time while waiting for a contributor to take it up but there does not seem to have been anyone that did so. So we are going to close this issue to clear up the queues and make it easier for contributors to browse possible implementation targets.

However, we're absolutely always up for discussion. Because this project is very active, we're unlikely to see comments made on closed tickets and we lock them after some time. If you or anyone else has any further questions, please let us know by using any of the communication methods listed in the page below:

• https://docs.ansible.com/ansible/latest/community/communication.html

In the future, sometimes starting a discussion on the development list prior to proposing or implementing a feature can make getting things included a little easier, but it's not always necessary.

Thank you once again for this and your interest in Ansible!

click here for bot help