loading data as yaml with date or datetime fails with json exception

[epDHowwD]

some data file:

example:
    - someDate: 2022-02-02

Using the cli interface:

$ jinja --version
jinja, version 1.0.7

Error is:

$ jinja render -t foobar -d data.yaml
Traceback (most recent call last):
  File "/usr/local/bin/jinja", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/jinja2_tools/cli.py", line 53, in render
    data = Data(data, verbose).get_data()
  File "/usr/local/lib/python3.10/site-packages/jinja2_tools/objects.py", line 69, in get_data
    print_verbose({'title': '[Data]', 'content': json.dumps(
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type date is not JSON serializable

Looking into objects.py there is a json.dumps invocation on the loaded yaml which seems not be able to cope with date and timestamps.

[Avielyo10]

@epDHowwD Hi 👋
Apologies for the delay in addressing your concern. JSON has specific guidelines regarding the allowable types, which include Number, String, Boolean, Array, Object, Whitespace, and Null.

Try:

example:
    - someDate: "2022-02-02"

[epDHowwD]

No I cannot change the database so the tool on top works. The yaml data comes various data sources and we use your cmdline tool (and I cannot underline its usefulness in this scenario enough) to extract information from it to display it in our documentation. We basically generate documentation from the actual configuration files. These config files we cannot change just so they work in the docu display well.

What we did, and what I propose you to fix differently, is to patch the tool in the way to remove the verbosity logging as json of the data being read in the function jinja2_tools.objects.Data.get_data:

#!/usr/bin/env python3

# -*- coding: utf-8 -*-
import re
import sys
import jinja2_tools

# fixes https://github.com/Avielyo10/jinja2-tools/issues/3

from jinja2_tools.exceptions import InvalidDataType
from jinja2_tools.util import input_handler, load_yaml
from jinja2_tools.cli import main

def fixed_get_data(self):
    try:
        ih_content = input_handler(self.data)
    except InvalidDataType as err:
        print(err.message, file=sys.stderr)
        sys.exit(128)
    else:
        self.data = load_yaml(ih_content)
        # print_verbose({'title': '[Data]', 'content': json.dumps(
        #     self.data, indent=2), 'verbose': self.verbose})
        return self.data

jinja2_tools.objects.Data.get_data = fixed_get_data

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

For example you can replace the json.dumps function with some yaml dump or pprint. That would cover more data types. At least as much as yaml supports generally.

[Avielyo10]

@epDHowwD got you, I'll create a fix for all the dumps!
Thanks!

[Avielyo10]

Done!
Thank you for using this tool!