Problem using loop.index when iterating over itertools.groupby result

[binrush]

Hello. I have strange jinja2 behavior when iterating over result produced by itertoos.groupby function. When I don't use loop.index, everything is OK:

import operator
import itertools
from jinja2 import Template

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))
template = """
  {% for g in i %}
    {% for e in g[1] %}
      {{ e }}
  {% endfor %}
{% endfor %}
"""
tpl = Template(template)
print tpl.render(i=i)

Output is: (1, 'a') (1, 'b') (2, 'c') (3, 'd'), so loop iterates over all elements.

When I call loop.index in loop, I have only last element in output:

import operator
import itertools
from jinja2 import Template

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))
template = """
  {% for g in i %}
    {% for e in g[1] %}
      {{ e }} {{loop.index}}
  {% endfor %}
{% endfor %}
"""
tpl = Template(template)
print tpl.render(i=i)

Output is (3, 'd') 1

Is it a bug or feature?

P.S. I'm using Jinja2 2.8 and python 2.7

[rockwelln]

waw, I found this issue a good study case :-).
Actually, it comes from the implementation of the LoopContext(Iterator).
Because itertools.groupbyreturn a generator which yield nested generators on the same collection, it needs to be fetched in order (see https://docs.python.org/2/library/itertools.html#itertools.groupby for details).

But when you use the 'special variable' loop, the collection is enclosed into a LoopContext instance.
The LoopContext class (and LoopContextIterator) try to prefetch the next element in the collection and return the current one. Most of the time it works and it's safe but here it creates the following sequence:

import operator
import itertools

l = [(1, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
i = itertools.groupby(l, lambda e: operator.getitem(e, 0))

>>> e, e_next = next(i), next(i)
>>> list(e[1])
[]
>>> e, e_next = e_next, next(i)
>>> list(e[1])
[]
>>> e, e_next = e_next, next(i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> list(e_next[1])  # detects the end of the collection and return the current element.
[(3, 'd')]

For me, it's a bug but this change doesn't seem so trivial to fix.