Skip to content

Loading…

Macro called too early? #148

Open
wulfraem opened this Issue · 1 comment

2 participants

@wulfraem

Hi, I'm having some troubles with calling macros.
I'm calling a macro and pass a parameter to it.
Inside the macro operations are performed on the given object
and it seems that the macro is called earlier than I would assume it would.

Example macro:
{%- macro testMacro(list) %}
{%- do list.append('1') %}
{%- endmacro %}

Example template:
{%- import 'TestMacro.html' as test with context %}

{%- set myList = [] %}
{{ myList }}

{%- do myList.append('a') %}
{{ myList }}

{%- do myList.append('b') %}
{{ myList }}

{{ test.testMacro(myList) }}
{{ myList }}

Output:
[]
['a']
['a', 'b', '1']
['a', 'b', '1']

As you can see, the macro is called before the second output.
Can I some enforce a "serial" behaviour? As I would like the output to be:
[]
['a']
['a', 'b']
['a', 'b', '1']

@soulseekah

Allow me to take a simpler case (without imports, etc.):

template = """
{% macro add_list( list ) %}
    {% do my_list.append( 3 ) %}
    {{ list }}
{% endmacro %}

{% set my_list = [ 1 ] %}
{{ my_list }}
{{ add_list( my_list ) }}
{{ add_list( my_list ) }}
{{ add_list( my_list ) }}
"""

The above template is tokenized top-to-bottom.

e = Environment( extensions=['jinja2.ext.do'] )
tokens = e._tokenize( template, None, None )
while tokens.current[1] != 'eof':
    print repr( tokens.current )
    tokens.next()

Token(1, 'data', u'\n')
Token(2, 'block_begin', u'{%')
Token(2, 'name', 'macro')
Token(2, 'name', 'add_list')
Token(2, 'lparen', u'(')
--- snip ---
Token(11, 'rparen', u')')
Token(11, 'variable_end', u'}}')
Token(11, 'data', u'\n')

These tokens are parsed one by one inside here https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/parser.py#L844

You can see there that variables are added as data https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/parser.py#L859

You can look at what subparse returns (especially in what order) by doing this:

p = Parser( e, template )
b = p.subparse()
print repr( b )

This then gets passed to https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/nodes.py#L252 nodes.Template, where the whole node list is wrapped into one Template node. Then it's maybe optimized https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L476 and finally compiled https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L902 by https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/compiler.py#L57

You can look at what Python source code the compiler produced by doing this:

from jinja2 import Environment
from jinja2.parser import Parser
from jinja2.compiler import generate

e = Environment( extensions=['jinja2.ext.do'] )
s = Parser( e, template ).parse()
print generate( p, e, None, None )

You can also view it by calling Environment.compile with the raw argument set to True. The is then compiled with the standard built-in function to bytecode, which is then executed by https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L834 when the Environment returns a Template.from_string. You can do this by repeating that code and doing something like:

c = generate( p, e, None, None )
namespace = { 'environment': e, '__file__': '<template>' }
exec c in namespace

Where namespace will contain the root function and the other variables available locally. The Template then does this https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L857 with the new namespace, setting the root_render_func to the root function in the render code. And then it all gets executed nicely when you render your template, https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L891 and by passing it a new Context https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/runtime.py#L50 you can make it run in isolation and step through it as needed.

from jinja2.runtime import new_context
new_context( e, '<template>', {} )
namespace['root']( ctx )

TL;DR a.k.a The Important Bits

This is the code generated:

from __future__ import division
from jinja2.runtime import LoopContext, TemplateReference, Macro, Markup, TemplateRuntimeError, missing, concat, escape, markup_join, unicode_join, to_string, identity, TemplateNotFound
name = None

def root(context, environment=environment):
    if 0: yield None
    yield u'\n'
    def macro(l_list):
        t_1 = []
        l_my_list = context.resolve('my_list')
        pass
        t_1.append(
            u'\n\t',
        )
        context.call(environment.getattr(l_my_list, 'append'), 3)
        t_1.extend((
            u'\n\t',
            to_string(l_list),
            u'\n',
        ))
        return concat(t_1)
    context.exported_vars.add('add_list')
    context.vars['add_list'] = l_add_list = Macro(environment, macro, 'add_list', ('list',), (), False, False, False)
    yield u'\n\n'
    l_my_list = [1]
    context.vars['my_list'] = l_my_list
    context.exported_vars.add('my_list')
    yield u'\n%s\n%s\n%s\n%s\n' % (
        l_my_list,
        context.call(l_add_list, l_my_list),
        context.call(l_add_list, l_my_list),
        context.call(l_add_list, l_my_list),
    )

blocks = {}
debug_info = '2=8&3=15&4=18&7=25&8=29&9=30&10=31&11=32'

As you can see it's pretty much "serial", apart from one bit: the string placeholders. Try the following:

def a( l ):
    l.append( 4 )
    return l
l = list()
print '%s %s %s' % ( a( l ), a( l ), a( l ) )
# Output: [4, 4, 4, 4, 4] [4, 4, 4, 4, 4] [4, 4, 4, 4, 4]

So that's pretty much where the issue lies. That's how string formatting works in Python (both % formatting and format), the l is shared and the string is baked only after all functons have been evaluating. But each function (macro) alters the result of the previous one.

Now that we know what goes where, it has to be decided whether this is a popular-enough edge case or not for a patch to be written and eventually accepted. Hope this helps and makes sense.

On another note, templates shouldn't really be doing all this macro-dancing around, should they? Keep them as straight-forward as possible and keep all logic in the view code. At least that's what I heard they do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.