Replace `jsonify()` use(s), accounting for `json.dumps()` differences

[moreati]

Prior to removal/restorationansible.parsing.utils.jsonify.jsonify() changed very little. Differences to json.dumps()

1. jsonify(None) -> '{}' vs dumps(None) -> 'null'
2. jsonify(..., format: bool) vs dumps(..., indent: int|None, ...)
3. jsonify() always applies sort_keys=True.
4. jsonify() prefers ensure_ascii=False, falling back to ensure_ascii=True on UnicodeDecodeError. I'm unsure what would trigger that exception only in the former case.

Refs

1. https://github.com/ansible/ansible/blame/a0495fc31497798a7a833ba7406a9729e1528dd8/lib/ansible/parsing/utils/jsonify.py
2. https://docs.python.org/3/library/json.html#basic-usage

Originally posted by @moreati in #1258

Edit 2025-06-17: Corrected return types when None is passed. Both return str on Python 2.x, and on Python 3.x.

[moreati]

4. jsonify() prefers ensure_ascii=False, falling back to ensure_ascii=True on UnicodeDecodeError. I'm unsure what would trigger that exception only in the former case.

Python 2.7 json.dumps() docs say (emphasis mine)

If ensure_ascii is false, the result may contain non-ASCII characters and the return value may be a unicode instance.

Python 3.0 docs say (emphasis mine) ETA: This is almost certainly a documentation bug, unicode type doesn't exist in Python 3.x

If ensure_ascii is False, then the return value will be a unicode instance.

Python 3.13 json.dumps() docs don't mention ensure_ascii, delegating everything to json.dump() docs

If True (the default), the output is guaranteed to have all incoming non-ASCII characters escaped. If False, these characters will be outputted as-is.

[moreati]

Python 3.0 doc also incorrectly lists an encoding argument.

[moreati]

@stefanor do you have an example of a playbook that failed when jsonify() was replaced with json.dumps()?

[stefanor]

I can't reproduce it any more, but I recall an empty list being turned into [None] or something like that.

The issue must have been resolved by either a change in ansible or mitogen :(

[moreati]

I can't reproduce it any more, but I recall an empty list being turned into [None] or something like that.

Thanks, I'll keep an eye out.

[moreati]

4. jsonify() prefers ensure_ascii=False, falling back to ensure_ascii=True on UnicodeDecodeError. I'm unsure what would trigger that exception only in the former case.

Found an inverse case (ensure_ascii=False succeeds, ensure_ascii=True raises) case on Python 2

>>> import sys, json
>>> sys.version_info
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
>>> s = json.dumps(b'\xff', ensure_ascii=False)
>>> s
'"\xff"'
>>> s = json.dumps(b'\xff')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 201, in encode
    return encode_basestring_ascii(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: invalid start byte

In Python 3 json.dumps(any_bytes_obj) raises TypeError.

[moreati]

4. jsonify() prefers ensure_ascii=False, falling back to ensure_ascii=True on UnicodeDecodeError. I'm unsure what would trigger that exception only in the former case.

I think the exception handling was first added in ansible/ansible@4fafd3b. The commit message doesn't illuminate matters much. The function moved from ansible.utils to ansible.parsing.utils.jsonify during the development of Ansible 2.0.

Based on

git log -G'json.dumps\(result2?, sort_keys=True, indent=indent, ensure_ascii=False\)'

[moreati]

4. jsonify() prefers ensure_ascii=False, falling back to ensure_ascii=True on UnicodeDecodeError. I'm unsure what would trigger that exception only in the former case.

claude.ai (with some nudging) found a case on Stackoverflow

>>> sys.version
'2.7.16 (v2.7.16:413a49145e, Mar  2 2019, 14:32:10) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]'

>>> s = json.dumps([u'\xc4', b'\xc3\x84'], ensure_ascii=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 210, in encode
    return ''.join(chunks)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

>>> s = json.dumps([u'\xc4', b'\xc3\x84'])
>>> s
'["\\u00c4", "\\u00c4"]'

• https://stackoverflow.com/questions/33118954/why-does-python-json-dumps-fail-on-mixed-utf-8-unicode-strings