TypeError: a bytes-like object is required (ansible_modlib.zip/ansible/module_utils/basic.py) #17038

DSpeichert · 2016-08-10T21:24:58Z

ISSUE TYPE

Bug Report

COMPONENT NAME

General Ansible / no idea

ANSIBLE VERSION

ansible 2.1.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

None

OS / ENVIRONMENT

Plain install of Ansible Tower on CentOS 7, up to date

SUMMARY

TypeError in ansible-generated script

STEPS TO REPRODUCE

Unsure which part of playbook may be relevant

EXPECTED RESULTS

Works when run manually on Ansible 2.1.1.0

ACTUAL RESULTS

Identity added: /tmp/ansible_tower_j61zhf/credential (/tmp/ansible_tower_j61zhf/credential)
Using /var/lib/awx/projects/_4__demo_project/ansible.cfg as config file

PLAYBOOK: chmurka.yml **********************************************************
3 plays in chmurka.yml

PLAY [Set up CIAPI] ************************************************************

TASK [setup] *******************************************************************
<api-host> ESTABLISH SSH CONNECTION FOR USER: root
<api-host> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/tmp/ansible_tower_j61zhf/cp/ansible-ssh-%h-%p-%r api-host '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1470863668.41-135499919505075 `" && echo ansible-tmp-1470863668.41-135499919505075="` echo $HOME/.ansible/tmp/ansible-tmp-1470863668.41-135499919505075 `" ) && sleep 0'"'"''
<api-host> PUT /tmp/tmp9PpGTu TO /root/.ansible/tmp/ansible-tmp-1470863668.41-135499919505075/setup
<api-host> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/tmp/ansible_tower_j61zhf/cp/ansible-ssh-%h-%p-%r '[api-host]'
<api-host> ESTABLISH SSH CONNECTION FOR USER: root
<api-host> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/tmp/ansible_tower_j61zhf/cp/ansible-ssh-%h-%p-%r -tt api-host '/bin/sh -c '"'"'LANG=en_US.utf8 LC_ALL=en_US.utf8 LC_MESSAGES=en_US.utf8 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1470863668.41-135499919505075/setup; rm -rf "/root/.ansible/tmp/ansible-tmp-1470863668.41-135499919505075/" > /dev/null 2>&1 && sleep 0'"'"''
fatal: [api-host]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "setup"}, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_uoiph15r/ansible_module_setup.py\", line 127, in <module>\r\n    main()\r\n  File \"/tmp/ansible_uoiph15r/ansible_module_setup.py\", line 117, in main\r\n    supports_check_mode = True,\r\n  File \"/tmp/ansible_uoiph15r/ansible_modlib.zip/ansible/module_utils/basic.py\", line 679, in __init__\r\n  File \"/tmp/ansible_uoiph15r/ansible_modlib.zip/ansible/module_utils/basic.py\", line 1615, in _log_invocation\r\n  File \"/tmp/ansible_uoiph15r/ansible_modlib.zip/ansible/module_utils/basic.py\", line 475, in heuristic_log_sanitize\r\nTypeError: a bytes-like object is required, not 'str'\r\n", "msg": "MODULE FAILURE", "parsed": false}

NO MORE HOSTS LEFT *************************************************************
    to retry, use: --limit @chmurka.retry

PLAY RECAP *********************************************************************
api-host              : ok=0    changed=0    unreachable=0    failed=1

The text was updated successfully, but these errors were encountered:

DSpeichert · 2016-08-10T21:31:58Z

False alarm.

For future reference: it is a symptom of using Python 3 on the target host. My inventory variables weren't properly loaded and it defaulted to python3. Makes sure to use ansible_python_interpreter=python2.

ssbarnea · 2016-12-19T22:37:05Z

Sorry but this is a valid py3 compatibility bug and should not be closed just because there is a workaround for it.

  File "/Users/ssbarnea/os/ansible-lint/.tox/py35/lib/python3.5/site-packages/ansible/module_utils/splitter.py", line 90, in split_args
    items = args.split('\n')
TypeError: a bytes-like object is required, not 'str'

DSpeichert · 2016-12-19T22:49:37Z

@ssbarnea I'm not sure what are the target goals at the moment for Ansible to decide that, so I'll reopen and let project members decide.

ssbarnea · 2016-12-19T23:07:12Z

Thank you. I will make a PR, usually py3 bugs can easily be fixed, the only problem is their number.

bcoca · 2016-12-20T02:22:47Z

many py3 compatibility fixes have gone into Ansible 2.2, can you confirm this is still an issue in that or devel?

abadger · 2016-12-20T04:52:32Z

2.2.0 is where we got far enough into porting that we expect things to work. So if it fails on 2.2 or devel we'd look at fixing it there.

ssbarnea · 2016-12-20T16:24:17Z

Here is a clear py3 compatibility bug that I hit https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/filter/core.py#L207 -- long does not exist on py3.

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>

abadger · 2017-02-03T18:23:17Z

Taking a look at the various problems reported in this bug:

line 475, in heuristic_log_sanitize\r\nTypeError: a bytes-like object is required, not 'str'\r\n",
- This seems to be caused by the data parameter to heuristic_log_sanitize() being passed in as bytes but then string literals are used within the function to operate on the data. The current ansible-2.2.x and devel remedy this by transforming the data parameter into a native string at the very top of the method: data = to_native(data).
the long bug in core.py was fixed by ssbarnea py3 compatibility fix reported on #17038 #19569

The splitter problem... If it's still happening in 2.2 or devel I think we should open a separate bug for it. A reproducer and a complete traceback would help enormously. I've just looked at the code and the text/bytes conversions don't make complete sense to me. Having the traceback and reproducer will let me figure out what is triggering this and I can talk to jimi about whether there's a reason the code is currently doing doing things the way it is.

anisse · 2017-04-11T11:40:24Z

It would have been nice to cherry-pick the fix in a bugfix release. Right now 2.2.2.0 users are in the cold for this bug (see #20257 ), having to install from the devel branch.

Edit: Nevermind, because #19930 is WONTFIX, this won't really be of any help.

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com> (cherry picked from commit 4ebd763)

abadger · 2017-04-11T16:32:00Z

@anisse, if you'd like to reach out to me (abadger1999) on irc.freenode.net I'd be happy to look at fixes that should be cherry-picked. I do see that 19569 wasn't cherry-picked to stable-2.2 so I've cherry-picked it now in case we have another 2.2.x release.

Note that #19930 is won't fix because it is an incompatibility with python-3.4 and should work fine with python-3.5. We only support python-3.5 and greater which should be of help to most (but unfortunately not all) of what people are using in the wild.

greggilbert · 2017-04-13T20:51:22Z

Sorry to hijack this closed thread, but I'm still having this issue when using ec2_facts:

  - name: Gather facts
    action: ec2_facts

TASK [build-run : Gather facts] *********************************************************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: a bytes-like object is required, not 'str'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_26vc1yju/ansible_module_ec2_facts.py\", line 195, in <module>\n    main()\n  File \"/tmp/ansible_26vc1yju/ansible_module_ec2_facts.py\", line 184, in main\n    ec2_facts = Ec2Metadata(module).run()\n  File \"/tmp/ansible_26vc1yju/ansible_module_ec2_facts.py\", line 167, in run\n    self.fetch(self.uri_meta)  # populate _data\n  File \"/tmp/ansible_26vc1yju/ansible_module_ec2_facts.py\", line 123, in fetch\n    subfields = raw_subfields.split('\\n')\nTypeError: a bytes-like object is required, not 'str'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}

For reference, my setup:

$ /usr/bin/python3 --version
Python 3.5.2

$ ansible --version
ansible 2.3.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609]

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

# the inventory file I'm using
[all]
localhost

[all:vars]
ansible_python_interpreter=/usr/bin/python3

abadger · 2017-04-13T21:34:27Z

@greggilbert Please open a new issue for the following reasons:

We don't actively look at closed and merged bugs/PRs so it's only by chance that your reply to one will be seen by someone who can do anything about it.
Your error is different than this one. This one was occurring in the basic.py code whereas yours is occurring in ec2_facts.py. Thanks!

DSpeichert closed this as completed Aug 10, 2016

DSpeichert reopened this Dec 19, 2016

ansibot added affects_2.1 This issue/PR affects Ansible v2.1 bug_report needs_triage Needs a first human triage before being processed. labels Dec 20, 2016

bcoca added needs_info This issue requires further information. Please answer any outstanding questions. and removed needs_triage Needs a first human triage before being processed. labels Dec 20, 2016

ssbarnea added a commit to ssbarnea/ansible that referenced this issue Dec 20, 2016

py3 compatibility fix reported on ansible#17038

df072a5

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>

nitzmahone pushed a commit that referenced this issue Dec 22, 2016

py3 compatibility fix reported on #17038 (#19569)

4ebd763

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>

alikins added the python3 label Jan 10, 2017

abadger closed this as completed Feb 3, 2017

abadger pushed a commit that referenced this issue Apr 11, 2017

py3 compatibility fix reported on #17038 (#19569)

6ee3a42

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com> (cherry picked from commit 4ebd763)

greggilbert mentioned this issue Apr 13, 2017

ec2_facts throws str error in Python 3.5+ #23595

Closed

sebastiaopf mentioned this issue May 18, 2017

Unarchive task fails second time it's ran #24781

Closed

ansibot removed the bug_report label Mar 7, 2018

ansibot added the bug This issue/PR relates to a bug. label Mar 7, 2018

ansible locked and limited conversation to collaborators Apr 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError: a bytes-like object is required (ansible_modlib.zip/ansible/module_utils/basic.py) #17038

TypeError: a bytes-like object is required (ansible_modlib.zip/ansible/module_utils/basic.py) #17038

DSpeichert commented Aug 10, 2016

DSpeichert commented Aug 10, 2016

ssbarnea commented Dec 19, 2016

DSpeichert commented Dec 19, 2016

ssbarnea commented Dec 19, 2016

bcoca commented Dec 20, 2016

abadger commented Dec 20, 2016

ssbarnea commented Dec 20, 2016

abadger commented Feb 3, 2017

anisse commented Apr 11, 2017 •

edited

abadger commented Apr 11, 2017

greggilbert commented Apr 13, 2017

abadger commented Apr 13, 2017

TypeError: a bytes-like object is required (ansible_modlib.zip/ansible/module_utils/basic.py) #17038

TypeError: a bytes-like object is required (ansible_modlib.zip/ansible/module_utils/basic.py) #17038

Comments

DSpeichert commented Aug 10, 2016

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

DSpeichert commented Aug 10, 2016

ssbarnea commented Dec 19, 2016

DSpeichert commented Dec 19, 2016

ssbarnea commented Dec 19, 2016

bcoca commented Dec 20, 2016

abadger commented Dec 20, 2016

ssbarnea commented Dec 20, 2016

abadger commented Feb 3, 2017

anisse commented Apr 11, 2017 • edited

abadger commented Apr 11, 2017

greggilbert commented Apr 13, 2017

abadger commented Apr 13, 2017

anisse commented Apr 11, 2017 •

edited