ansible 2.6 / python3: CallError: builtins.TypeError: the JSON object must be str, not 'bytes'

[candlerb]

• mitogen master plus Use lxc exec --mode=noninteractive which is more widely compatible #372 patch
• ubuntu 16.04.5, lxd 2.0.11
• python 2.7.9 on host, python 3.5.1 in lxd container
• ansible 2.6.4 from ppa

Test case:

==> test.yml <==
- hosts: gold-master
  tasks:
    - raw: echo hello world

==> hosts.yml <==
all:
  vars:
    ansible_python_interpreter: /usr/bin/python3
  hosts:
    gold-master:
      ansible_connection: lxd

==> ansible.cfg <==
[defaults]
inventory = hosts.yml
#strategy_plugins = /usr/local/share/mitogen/ansible_mitogen/plugins/strategy
#strategy = mitogen_linear

gold-master is an ubuntu 16.04 lxd container.

Works with mitogen commented out. With mitogen enabled, I get:

$ ansible-playbook test.yml

PLAY [gold-master] ******************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:     s.__class__.__name__))
fatal: [gold-master]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}
	to retry, use: --limit @/home/ubuntu/ansible/misc/test.retry

PLAY RECAP **************************************************************************************************************************
gold-master                : ok=0    changed=0    unreachable=0    failed=1

With -vvv there is an exception traceback shown:

[pid 3603] 11:13:12.040260 D mitogen: mitogen.core.Stream(u'unix_client.3622').on_disconnect()
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ansible/executor/task_executor.py", line 139, in run
    res = self._execute()
  File "/usr/lib/python2.7/dist-packages/ansible/executor/task_executor.py", line 584, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/local/share/mitogen-candlerb-371/ansible_mitogen/mixins.py", line 116, in run
    return super(ActionModuleMixin, self).run(tmp, task_vars)
  File "/usr/lib/python2.7/dist-packages/ansible/plugins/action/normal.py", line 46, in run
    result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
  File "/usr/local/share/mitogen-candlerb-371/ansible_mitogen/mixins.py", line 334, in _execute_module
    timeout_secs=self.get_task_timeout_secs(),
  File "/usr/local/share/mitogen-candlerb-371/ansible_mitogen/planner.py", line 484, in invoke
    kwargs=planner.get_kwargs(),
  File "/usr/local/share/mitogen-candlerb-371/ansible_mitogen/connection.py", line 428, in call
    return self._rethrow(recv)
  File "/usr/local/share/mitogen-candlerb-371/ansible_mitogen/connection.py", line 415, in _rethrow
    return recv.get().unpickle()
  File "/usr/local/share/mitogen-candlerb-371/mitogen/core.py", line 595, in unpickle
    raise obj
CallError: builtins.TypeError: the JSON object must be str, not 'bytes'
  File "<stdin>", line 1993, in _dispatch_one
  File "master:/usr/local/share/mitogen-candlerb-371/ansible_mitogen/target.py", line 353, in run_module
    impl = klass(**kwargs)
  File "master:/usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py", line 633, in __init__
    super(NewStyleRunner, self).__init__(**kwargs)
  File "master:/usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py", line 578, in __init__
    super(ScriptRunner, self).__init__(**kwargs)
  File "master:/usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py", line 445, in __init__
    super(ProgramRunner, self).__init__(**kwargs)
  File "master:/usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py", line 252, in __init__
    self.args = json.loads(json_args)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))

Looks like a bytes to str conversion is missing somewhere.

I tried a band-aid of json.loads(str(json_args)), but this then gives a different error UPDATE: see below.

[RonnyPfannschmidt]

please try to show the content of the json-value as well

[candlerb]

It was my mistake, I should have used decode() not str().

With the following sticking plaster it works:

--- /usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py.orig	2018-09-18 11:21:52.414854084 +0000
+++ /usr/local/share/mitogen-candlerb-371/ansible_mitogen/runner.py	2018-09-18 11:56:20.106790770 +0000
@@ -249,7 +249,7 @@
         self.service_context = service_context
         self.econtext = econtext
         self.detach = detach
-        self.args = json.loads(json_args)
+        self.args = json.loads(json_args.decode("UTF-8"))
         self.temp_dir = temp_dir
         self.extra_env = extra_env
         self.env = env

However, I imagine that really it's the responsibility of the caller of this function to pass json_args as as a string, not bytes.

FWIW, I did also log json_args. The value was:

b'{"_ansible_version": "2.6.4", "_ansible_selinux_special_fs": ["fuse", "nfs", "vboxsf", "ramfs", "9p"], "_ansible_no_log": false, "gather_timeout": 10, "_ansible_module_name": "setup", "_ansible_debug": false, "_ansible_verbosity": 3, "_ansible_keep_remote_files": false, "_ansible_syslog_facility": "LOG_USER", "_ansible_socket": null, "gather_subset": "all", "_ansible_diff": false, "_ansible_remote_tmp": "~/.ansible/tmp", "_ansible_shell_executable": "/bin/sh", "_ansible_check_mode": false, "_ansible_tmpdir": "/root/.ansible/tmp/ansible_mitogen_l9nzvee2/worker-4388-7ff4cc7b1650"}'

[dw]

Thanks for reporting this. I'll add something to CI, because that change needs tested in both directions.

[dw]

The problem is specific to Python>2,<3.6:

Changed in version 3.6: s can now be of type bytes or bytearray. The input encoding should be UTF-8, UTF-16 or UTF-32.

This is a minor pain, I need to add another CI target to cover 3.5. There is an annoying balance between speedy but inaccurate or dog-slow (and thus forgotten/ignored) CI

[dw]

Sorry for the crazy delay - discovered a can of worms around Python 3.x in the process! You may wish to consider subscribing to #426 in the meantime.

The fix for this was committed in 16911c9.

This is now on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://networkgenomics.com/mail/mitogen-announce/

Thanks for reporting this!