toml inventory cannot dump unsafe values #71307

freeseacher · 2020-08-17T14:19:22Z

SUMMARY

Rendering string

config:
      service: !unsafe '{{ SYSLOG_IDENTIFIER }}'

with

- name: place config
  copy:
    content: |
      {{ config | to_toml }}
    dest: "{{ config_file }}"

leads to

service = [ "{", "{", " ", "S", "Y", "S", "L", "O", "G", "_", "I", "D", "E", "N", "T", "I", "F", "I", "E", "R", " ", "}", "}",]

instead of

service = "{{ SYSLOG_IDENTIFIER }}"

I understand that filter to_toml is not part of ansible and get from https://github.com/sivel/toiletwater but in code of it it uses ansible code in import section.
https://github.com/sivel/toiletwater/blob/master/plugins/filter/toml.py#L10

ISSUE TYPE

Bug Report

COMPONENT NAME

to_toml

ANSIBLE VERSION

 % ansible --version
ansible 2.9.11
  config file = None
  configured module search path = ['/home/ashirokih/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ashirokih/python/lib/python3.8/site-packages/ansible
  executable location = /home/ashirokih/python/bin/ansible
  python version = 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.0]

CONFIGURATION

empty

OS / ENVIRONMENT

not relevant

STEPS TO REPRODUCE

Rendering string

config:
      service: !unsafe '{{ SYSLOG_IDENTIFIER }}'

with

- name: place config
  copy:
    content: |
      {{ config | to_toml }}
    dest: "{{ config_file }}"

EXPECTED RESULTS

service = "{{ SYSLOG_IDENTIFIER }}"

ACTUAL RESULTS

service = [ "{", "{", " ", "S", "Y", "S", "L", "O", "G", "_", "I", "D", "E", "N", "T", "I", "F", "I", "E", "R", " ", "}", "}",]

Probably i should ask @sivel

The text was updated successfully, but these errors were encountered:

sivel · 2020-08-17T14:30:35Z

The AnsibleTomlEncoder needs to be updated to handle AnsibleUnsafe. I'll submit a PR momentarily.

freeseacher · 2020-08-17T14:45:45Z

Any hints how i can apply that patch it in current version ? place toml.py to library dir ?

sivel · 2020-08-17T14:47:57Z

@freeseacher no, that will not be possible. It could be handled in toml.py from sivel.toiletwater by redefining toml_dumps and associated code directly there, instead of using what is shipped with Ansible, as a short term fix.

freeseacher · 2020-08-17T15:00:50Z

for now

10c10,11
< from ansible.plugins.inventory.toml import HAS_TOML, toml, toml_dumps
---
> 
> from ansible.plugins.inventory.toml import HAS_TOML, toml
13a15
> from ansible.parsing.yaml.objects import AnsibleSequence, AnsibleUnicode
15a18,30
> from ansible.utils.unsafe_proxy import AnsibleUnsafeBytes, AnsibleUnsafeText
> 
> 
> class AnsibleTomlEncoder(toml.TomlEncoder):
>     def __init__(self, *args, **kwargs):
>         super(AnsibleTomlEncoder, self).__init__(*args, **kwargs)
>         # Map our custom YAML object types to dump_funcs from ``toml``
>         self.dump_funcs.update({
>             AnsibleSequence: self.dump_funcs.get(list),
>             AnsibleUnicode: self.dump_funcs.get(str),
>             AnsibleUnsafeBytes: self.dump_funcs.get(str),
>             AnsibleUnsafeText: self.dump_funcs.get(str),
>         })
16a32
> toml_dumps = functools.partial(toml.dumps, encoder=AnsibleTomlEncoder())

fixes my problem. will move to new ansible version when it will be available.

tnx.

…71307 (ansible#71309) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

…1307 (ansible#71309) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

…#71309)

…71309) (#71318) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

…71309) (#71317) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

v2.9.13 ======= Minor Changes ------------- - Updated network integration auth timeout to 90 secs. - ansible-test - Remove ``pytest < 6.0.0`` constraint for managed installations on Python 3.x now that pytest 6 is supported. - known_hosts - fix reference to non-existent parameter in example (ansible/ansible#71417) Security Fixes -------------- - The fix for CVE-2020-1736 has been reverted. Users are encouraged to specify a ``mode`` parameter in their file-based tasks when the files being manipulated contain sensitive data. - dnf - Previously, regardless of the ``disable_gpg_check`` option, packages were not GPG validated. They are now. (CVE-2020-14365) Bugfixes -------- - Confirmed commit fails with TypeError in IOS XR netconf plugin (ansible-collections/cisco.iosxr#74) - Fix an exit code for a non-failing playbook (ansible/ansible#71306) - Fix execution of the meta tasks 'clear_facts', 'clear_host_errors', 'end_play', 'end_host', and 'reset_connection' when the CLI flag '--flush-cache' is provided. - Fix statistics reporting when rescue block contains another block (issue ansible/ansible#61253). - Fixed Ansible reporting validate not supported by netconf server when enabled in netconf - (ansible-collections/ansible.netcommon#119). - TOML inventory - Ensure we register dump functions for ``AnsibleUnsafe`` to support dumping unsafe values. Note that the TOML format has no functionality to mark that the data is unsafe for re-consumption. (ansible/ansible#71307) - ansible-test units - fixed collection location code to work under pytest >= 6.0.0 - aws_acm_info - fix `KeyError` failure when retrieving keys with a `Failed` status (ansible-collections/community.aws#198) - cron - cron file should not be empty after adding var (ansible/ansible#71207) - mongodb_replicaset - fixes authentication to determine replicaset name (ansible-collections/community.mongodb#136). - powershell - fix escaping of strings that broken modules like fetch when dealing with special chars - ansible/ansible#62781 - powershell - fix the CLIXML parser when it contains nested CLIXML objects - ansible/ansible#69550 - psrp - Use native PSRP mechanism when copying files to support custom endpoints - setup - Add a null check for ``Win32_Bios.ReleaseData`` to avoid a failure when that value is not set - ansible/ansible#69736 - strftime filter - Input epoch is allowed to be a float (ansible/ansible#71257) - systemd - fixed chroot usage on new versions of systemd, that broke because of upstream changes in systemctl output - systemd - made the systemd module work correctly when the SYSTEMD_OFFLINE environment variable is set - zabbix_host - fixed inventory_mode key error, which occurs with Zabbix 4.4.1 or more (ansible/ansible#65304). - zabbix_proxy - fixed support for Zabbix 5.0

sivel self-assigned this Aug 17, 2020

ansibot added affects_2.9 This issue/PR affects Ansible v2.9 bug This issue/PR relates to a bug. needs_triage Needs a first human triage before being processed. python3 support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Aug 17, 2020

sivel added a commit to sivel/ansible that referenced this issue Aug 17, 2020

Allow the TOML inventory to dump unsafe. Fixes ansible#71307

7e3cac7

sivel changed the title ~~to_toml and unsafe string~~ toml inventory cannot dump unsafe values Aug 17, 2020

sivel removed the needs_triage Needs a first human triage before being processed. label Aug 17, 2020

sivel mentioned this issue Aug 17, 2020

Allow the TOML inventory to dump unsafe #71309

Merged

ansibot added the has_pr This issue has an associated PR. label Aug 17, 2020

sivel closed this as completed in #71309 Aug 17, 2020

sivel added a commit that referenced this issue Aug 17, 2020

Allow the TOML inventory to dump unsafe. Fixes #71307 (#71309)

9da8801

sivel added a commit to sivel/ansible that referenced this issue Aug 17, 2020

[stable-2.10] Allow the TOML inventory to dump unsafe. Fixes ansible#…

424cc77

…71307 (ansible#71309) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

sivel added a commit to sivel/ansible that referenced this issue Aug 17, 2020

[stable-2.9] Allow the TOML inventory to dump unsafe. Fixes ansible#7…

781375b

…1307 (ansible#71309) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

bcoca pushed a commit to bcoca/ansible that referenced this issue Aug 19, 2020

Allow the TOML inventory to dump unsafe. Fixes ansible#71307 (ansible…

3685ae1

…#71309)

relrod pushed a commit that referenced this issue Aug 27, 2020

[stable-2.9] Allow the TOML inventory to dump unsafe. Fixes #71307 (#…

35d3ed5

…71309) (#71318) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

relrod pushed a commit that referenced this issue Aug 27, 2020

[stable-2.10] Allow the TOML inventory to dump unsafe. Fixes #71307 (#…

3f41c76

…71309) (#71317) (cherry picked from commit 9da8801) Co-authored-by: Matt Martz <matt@sivel.net>

ansible locked and limited conversation to collaborators Sep 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

toml inventory cannot dump unsafe values #71307

toml inventory cannot dump unsafe values #71307

freeseacher commented Aug 17, 2020

sivel commented Aug 17, 2020

freeseacher commented Aug 17, 2020

sivel commented Aug 17, 2020

freeseacher commented Aug 17, 2020

toml inventory cannot dump unsafe values #71307

toml inventory cannot dump unsafe values #71307

Comments

freeseacher commented Aug 17, 2020

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

sivel commented Aug 17, 2020

freeseacher commented Aug 17, 2020

sivel commented Aug 17, 2020

freeseacher commented Aug 17, 2020