Document how to (and not NOT to) use Jinja in states

[whiteinge]

This is a perennial issue in IRC and other support channels:

Jinja is not Python, is missing many useful programming constructs, and has restrictive variable scoping -- which are all great things for a templating language. However, people routinely use Jinja (and Jinja macros) as though they are functions to perform custom complex logic, or string concatenation, or computations. The syntax is awful and it's impossible to debug.

However, custom execution modules take 20-seconds to write (minus the custom logic/concatenation/computation part), are distributed automatically to minions, and are Python.

Document:

• That Jinja is evaluated before the YAML (and thus, before the states are run).
• /srv/salt/_modules/my_custom_module.py
• {{ salt['my_custom_module.my_custom_function']() }}
• When to use Jinja macros. (Hint: as mini-templates to repeat blocks of strings with a few parameterized variables).

Refs #12446.

[whiteinge]

This should be added to the tutorials, linked to from the Jinja docs, and linked to from the State docs.

[jasonrm]

As I've been working with salt on-and-off for a while now this probably describes the insanity I've created in my states. I have one macro that ends something like this…

<snip a lot of previous states and jinja logic>
module-{{ module }}-pkg-dependencies_{{ key }}_{{ os }}:
    pkg.installed:
        {% for thing in value -%}
        {% for _key, _value in thing.iteritems() -%}
        - {{ _key }}: {{ _value }}
        {% endfor -%}
        {% endfor -%}
        - user: {{ salt['config.merge']('system:user') }}
        - require:
            - file: module-os
            {% if stack|length -%}
            - pkg: module-{{ module }}-pkg-dependencies_{{ stack[-1] }}_{{ os }}
            {%- endif %}
        - reload_modules: True
        - require_in:
            - file: module-{{ module }}-dependencies-pkg
            - file: module-{{ module }}-dependencies
                            {% do stack.append(key) %}
                            {% endfor %}
                        {% endfor %}
                    {% endif %}
                {% endif -%}
            {% endfor -%}
        {% endif -%}
    {% endfor -%}

{%- endmacro %}

Yeah… past me must have really hated the future me who has to fix that mess.

It's been my experience that once something seems insane, it's usually because I'm using the wrong tool for the task. As such, I'm currently looking at converting most of my complex states to the pyobjects renderer. Should I be looking into making custom execution modules instead?

[whiteinge]

@jasonrm great example! Thanks for sharing it here. :)

PyObjects is great if you prefer that syntax and you're right that it would save you from having to create a custom execution module. If you'd rather stick with Jinja, I'd suggest trying to envision a small Python function that gathers all the data points you need and spits back a data structure you can use in Jinja with just a simple loop or two.

IMO, the maintenance ideal to strive for is Jinja's only role should be to grab external data from various sources (calls to modules, Pillar, Grains, or data-only YAML or JSON files (e.g., {% import_json "defaults.json" as defaults %} ) and then loop over that data. Anything that requires more than that should be done in a custom execution module -- or a more logic-friendly renderer like PyObjects.

[jacobhammons]

I've added this information to the Understanding Jinja topic, https://docs.saltstack.com/en/latest/topics/jinja/index.html. Closing.