Support unicode output when dumping yaml #3808
Conversation
|
aos-ci-test |
filter_plugins/oo_filters.py
Outdated
| @@ -657,7 +657,7 @@ def to_padded_yaml(data, level=0, indent=2, **kw): | |||
| try: | |||
| transformed = yaml.dump(data, indent=indent, allow_unicode=True, | |||
| default_flow_style=False, | |||
| Dumper=AnsibleDumper, **kw) | |||
| Dumper=AnsibleDumper, **kw).decode('utf-8') | |||
There was a problem hiding this comment.
Sadly, this won't work across versions
Python 2
$ python2 -c "import yaml; print(yaml.dump({}, allow_unicode=True).decode('utf-8'))"
{}
Python 3
$ python3 -c "import yaml; print(yaml.dump({}, allow_unicode=True).decode('utf-8'))"
Traceback (most recent call last):
File "<string>", line 1, in <module>
AttributeError: 'str' object has no attribute 'decode
There was a problem hiding this comment.
A possible solution is using six.u():
$ python2 -c "import six, yaml; print(six.u(yaml.dump({}, allow_unicode=True)))
{}
$ python3 -c "import six, yaml; print(six.u(yaml.dump({}, allow_unicode=True)))
{}
mtnbikenc
force-pushed
the
unicode-decode
branch
from
92b21f9
to
9a38f01
Compare
mtnbikenc
changed the title
|
Wrapping ansible.compat.six in a try/except block because ansible.compat.six goes away with Ansible 2.4. |
|
aos-ci-test |
|
[test] |
|
[merge] |
|
I have a hard time believing this is not a flake. |
|
@mtnbikenc you may want to try a rebase in case this is a flake from an earlier merge. |
mtnbikenc
force-pushed
the
unicode-decode
branch
from
9a38f01
to
203630f
Compare
|
aos-ci-test |
|
[merge] |
|
Evaluated for openshift ansible test up to 203630f |
|
continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_openshift_ansible_extended_conformance_install/101/) (Base Commit: 0bac74c) |
|
|
aos-ci-test |
|
[merge] |
|
[merge] |
|
Evaluated for openshift ansible merge up to 203630f |
|
continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_openshift_ansible/151/) (Base Commit: 68b3156) |
|
This appears to have resulted in #3978 - what's the change I need to make to how my identity provider var is specified here: https://github.com/openshift/release/blob/master/cluster/test-deploy/data/vars.yaml#L39 Or alternatively, is this unrelated and just looks like it? |
|
@smarterclayton @mtnbikenc I think we have an issue with PyYAML using vs |
|
Will try locally. |
|
The ansible dumper doesn't have the same semantics - what would a proposed patch look like that I can try? |
|
It looks like the ansible dumper, at least via code invocation, doesn't like Unicode by default: But is happy with it's
I'll see if I can create a quick diff that should work for testing. |
|
Switching over to the new issue. |
When using
allow_unicode=Truein yaml.dump statements, if the output contains a unicode character it will cause the module to fail. Usingsix.uto properly pass through all unicode characters. This can have the side effect of passing unicode characters even if they were not intentionally included, such as non-breaking spaces in URLs. See bug 1416877.