Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Fix loop.last memory issue #92

Merged
merged 1 commit into from

2 participants

Jason Kotenko Armin Ronacher
Jason Kotenko

We are using a generator to input data to jinja2 templates. When we render, the data is streamed out using TemplateStream, which works great unless we have loop.last in our template.

The code for loop.last currently reads in the whole iterator (in our case a generator which reads files) in order to determine the length of the loop, and check to see if this is the last iteration.

There is an alternative to checking the length, which is to keep the next element in the iteration stored in memory, and checking that element for a sentinel value to see if the loop is on the last iteration.

That is the approach I took in this pull request. We are patching our jinja2 packages internally, in order to allow us to use loop.last on big data (though loop.length is still dangerous, and by extension, the repr method on LoopContext). Thanks.

Jason Kotenko Fixed loop.last to not consume the entire iterator to determine if th…
…is is

the last iteration of the loop.
776567c
Armin Ronacher mitsuhiko merged commit ea89feb into from
Joe Abbate jmafc referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 24, 2012
  1. Fixed loop.last to not consume the entire iterator to determine if th…

    Jason Kotenko authored
    …is is
    
    the last iteration of the loop.
This page is out of date. Refresh to see the latest.
Showing with 14 additions and 2 deletions.
  1. +14 −2 jinja2/runtime.py
16 jinja2/runtime.py
View
@@ -266,10 +266,12 @@ def __call__(self):
class LoopContext(object):
"""A loop context for dynamic iteration."""
+ End = object()
def __init__(self, iterable, recurse=None):
self._iterator = iter(iterable)
self._recurse = recurse
+ self._after = self._safe_next()
self.index0 = -1
# try to get the length of the iterable early. This must be done
@@ -288,7 +290,7 @@ def cycle(self, *args):
return args[self.index0 % len(args)]
first = property(lambda x: x.index0 == 0)
- last = property(lambda x: x.index0 + 1 == x.length)
+ last = property(lambda x: x._after is LoopContext.End)
index = property(lambda x: x.index0 + 1)
revindex = property(lambda x: x.length - x.index0)
revindex0 = property(lambda x: x.length - x.index)
@@ -299,6 +301,12 @@ def __len__(self):
def __iter__(self):
return LoopContextIterator(self)
+ def _safe_next(self):
+ try:
+ return next(self._iterator)
+ except StopIteration:
+ return self.End
+
@internalcode
def loop(self, iterable):
if self._recurse is None:
@@ -344,7 +352,11 @@ def __iter__(self):
def next(self):
ctx = self.context
ctx.index0 += 1
- return next(ctx._iterator), ctx
+ if ctx._after is LoopContext.End:
+ raise StopIteration
+ next_elem = ctx._after
+ ctx._after = ctx._safe_next()
+ return next_elem, ctx
class Macro(object):
Something went wrong with that request. Please try again.