Problems with non-ASCII strings from dynamic inventory #10007
Comments
|
I'll have to look through the git log around json_dict_to_bytes() to refresh my memory on this... I think if there is a testcase it would be an integrarion test rather than a unittest. The change is definitely going to cause problems... at the least, a regression of the problem that json_dict_to_bytes() was written for. But I haven't found a jinja2 method of processing non-ascii byte strings either. So I suspect some portion of the code is calling jinja templating and json_dict_to_bytes() in the opposite order of what they need to be. (the jinja templating needs to be finished before we call json_dict_to_bytes()). We'll have to look through the git log and code to see if that's easy to fix or extremely hard... |
|
Just an update here - Looks like the problem I wrote this for can be fixed at a specific point instead of all throughout the code. This will also make the code closer to the v2 tree (where we decided to use unicode type internally). To do that I'll add the transformation to a byte str in the callbacks.py function on_unreachable() and then remove the calls to json_dict_to_bytes() from the dynamic inventory handling. |
|
Thanks for the feedback. We're really looking forward to have this fixed. Or, either, v2 being published, as we always run with the latest Ansible release. |
|
So I just had someone who reported me the exact same bug during a workshop I gave, and I found a minimal test case: With: (use case was using vars_prompt and deploy people name in the git configuration, and the person had Unicode in his name). So at least, having instruction on how to use the v2 tree would help (in fact, I wonder why this is not a git branch, as this would ease test from community for small issue like this) Should it be treated as a different bug ? |
|
I think I have a partial fix -- it removes one of the two places where json_dict_to_bytes() is called (when getting the input from the dynamic inventory script) and adds an explicit encoding step to the callbacks.on_unreachable() callback. I'm just trying to determine:
|
|
@mscherer I believe that may be a different problem. I think this fix is just changing the interface between dynamic inventory scripts and the main part of ansible. to test this on v2, you should be able to source v2/hacking/env-setup just like you could source the toplevel hacking/env-setup. That will setup all of the paths to run the v2 version of ansible and ansible-playbook. It's not complete by any stretch but many of the integration tests are now passing. |
|
Okay, answer to my questions:
|
|
Okay, I think that patch should fix the problems with dynamic inventory. I do believe that the same sort of thing could happen with extra-vars as @mscherer is experiencing but we should probably track it separately -- it will be a different portion of the code that handles parsing of vars given via the command line that we'll need to modify. |
|
And of course, if the problem isn't fixed, feel free to open a new bug report or otherwise let me know! Thanks. Closing This TicketHi! We believe recent commits (likely detailed above) should resolve this question or problem for you. This will also be included in the next major release. If you continue seeing any problems related to this issue, or if you have any further questions, please let us know by stopping by one of the two mailing lists, as appropriate:
Because this project is very active, we're unlikely to see comments made on closed tickets, but the mailing list is a great way to ask questions, or post if you don't think this particular Thank you! |
…m potential byte str into unicode type. This fix is for dynamic inventory. Fixes #10007 Conflicts: v2/ansible/inventory/script.py v2/ansible/module_utils/basic.py
Issue Type:
Bug Report
Ansible Version:
1.8, 1.9, devel
Environment:
Ubuntu 14.04.1 LTS x86_64
Summary:
When using a dynamic inventory, if a variable definition contains non-ASCII characters (but correct UTF-8 sequences or even correct Unicode escapes
\u00e1it will later fail at whatever template expansion attepted: wheter it's from the playbook itself or through thetemplatemodule.The problem seems to be related to the fact that Jinja2 can only handle either Python unicode objects or plain ASCII strings, but not strings encoded in UTF-8 as it is receiving right now.
Steps To Reproduce:
Configure Ansible to use the following inventory script:
And then try to execute the following example playbook:
Expected Results:
Actual Results:
Notes:
As a temporary fix, we applied the following patch to our installation:
However, this simple change is semantically significant, even though it seems to pass the test suite (see below), we are not sure it is the proper place to fix it.
Test run:
The text was updated successfully, but these errors were encountered: