Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem using loop.index when iterating over itertools.groupby result #555

Closed
binrush opened this issue Mar 1, 2016 · 1 comment · Fixed by #1101
Closed

Problem using loop.index when iterating over itertools.groupby result #555

binrush opened this issue Mar 1, 2016 · 1 comment · Fixed by #1101
Labels
bug
Milestone

Comments

@binrush
Copy link

@binrush binrush commented Mar 1, 2016

Hello. I have strange jinja2 behavior when iterating over result produced by itertoos.groupby function. When I don't use loop.index, everything is OK:

import operator
import itertools
from jinja2 import Template

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))
template = """
  {% for g in i %}
    {% for e in g[1] %}
      {{ e }}
  {% endfor %}
{% endfor %}
"""
tpl = Template(template)
print tpl.render(i=i)

Output is: (1, 'a') (1, 'b') (2, 'c') (3, 'd'), so loop iterates over all elements.

When I call loop.index in loop, I have only last element in output:

import operator
import itertools
from jinja2 import Template

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))
template = """
  {% for g in i %}
    {% for e in g[1] %}
      {{ e }} {{loop.index}}
  {% endfor %}
{% endfor %}
"""
tpl = Template(template)
print tpl.render(i=i)

Output is (3, 'd') 1

Is it a bug or feature?

P.S. I'm using Jinja2 2.8 and python 2.7

@rockwelln

This comment has been minimized.

Copy link

@rockwelln rockwelln commented Oct 18, 2016

waw, I found this issue a good study case :-).
Actually, it comes from the implementation of the LoopContext(Iterator).
Because itertools.groupbyreturn a generator which yield nested generators on the same collection, it needs to be fetched in order (see https://docs.python.org/2/library/itertools.html#itertools.groupby for details).

But when you use the 'special variable' loop, the collection is enclosed into a LoopContext instance.
The LoopContext class (and LoopContextIterator) try to prefetch the next element in the collection and return the current one. Most of the time it works and it's safe but here it creates the following sequence:

import operator
import itertools

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))

>>> e, e_next = next(i), next(i)
>>> list(e[1])
[]
>>> e, e_next = e_next, next(i)
>>> list(e[1])
[]
>>> e, e_next = e_next, next(i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> list(e_next[1])  # detects the end of the collection and return the current element.
[(3, 'd')]

For me, it's a bug but this change doesn't seem so trivial to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants
You can’t perform that action at this time.