Optional literal eval on native environments

[irgolic]

I would like jinja2's NativeEnvironment to support rendering without passing string results through ast.literal_eval.

I ran into the unexpected behavior described in #1403, which was explained away as:

Note that the nativetypes functionality is effectively just literal_eval (docs).

Native types' functionality is much more different than just literal eval:

import pydantic
from jinja2 import Environment
from jinja2.nativetypes import NativeEnvironment

normal_env = Environment()
native_env = NativeEnvironment()


class MyModel(pydantic.BaseModel):
    foo: str


context = {
    "model": MyModel(foo="bar"),
    "num": 2,
    "string_num": "2",
}

# native returns non-stringified values
model = "{{ model }}"
print(type(normal_env.from_string(model).render(context)))  # <class 'str'>
print(type(native_env.from_string(model).render(context)))  # <class '__main__.MyModel'>

# native returns Undefined for missing keys
missing = "{{ nope }}"
print(type(normal_env.from_string(missing).render(context)))  # <class 'str'>
print(type(native_env.from_string(missing).render(context)))  # <class 'jinja2.runtime.Undefined'>

# native returns None for lone failed if block
empty = "{% if nope %}{{ nope }}{% endif %}"
print(type(normal_env.from_string(empty).render(context)))  # <class 'str'>
print(type(native_env.from_string(empty).render(context)))  # <class 'NoneType'>

# native returns int for the `num`
num = "{{ num }}"
print(type(normal_env.from_string(num).render(context)))  # <class 'str'>
print(type(native_env.from_string(num).render(context)))  # <class 'int'>

# native CASTS `string_num` to an int
string_num = "{{ string_num }}"
print(type(normal_env.from_string(string_num).render(context)))  # <class 'str'>
print(type(native_env.from_string(string_num).render(context)))  # <class 'int'>

Coming from ordinary jinja environments, it feels like NativeEnvironment respects the types of the objects in the context. Except for the literal_eval call at the end.

It would be great if this was somehow optional, e.g., toggleable via a kwarg.

For completeness, here's the workaround I'm relying on in place of this

from itertools import islice, chain
from types import GeneratorType

import jinja2.nativetypes
import typing as t


def custom_native_concat(values: t.Iterable[t.Any]) -> t.Optional[t.Any]:
    """
    An amended jinja2.nativetypes.native_concat that doesn't try to parse the output as a Python literal.
    """
    head = list(islice(values, 2))

    if not head:
        return None

    if len(head) == 1:
        raw = head[0]
        if not isinstance(raw, str):
            return raw
    else:
        if isinstance(values, GeneratorType):
            values = chain(head, values)
        raw = "".join([str(v) for v in values])

    # try:
    #     return literal_eval(
    #         # In Python 3.10+ ast.literal_eval removes leading spaces/tabs
    #         # from the given string. For backwards compatibility we need to
    #         # parse the string ourselves without removing leading spaces/tabs.
    #         parse(raw, mode="eval")
    #     )
    # except (ValueError, SyntaxError, MemoryError):
    #     return raw
    return raw


class NativeEnvironment(jinja2.nativetypes.NativeEnvironment):
    concat = staticmethod(custom_native_concat)


class NativeTemplate(jinja2.nativetypes.NativeTemplate):
    environment_class = NativeEnvironment


NativeEnvironment.template_class = NativeTemplate

[davidism]

here's the workaround I'm relying on

Using subclassing to achieve the behavior you want isn't a "workaround", it's a standard way to extend the behavior. I don't plan to change native environment at this time, and you can already accomplish what you need with the API available to you.