Added rewind() to TokenStream #194

wants to merge 1 commit into


None yet

2 participants

Kronuz commented Mar 22, 2013

I added rewind() to TokenStream. look() now uses it to get the next token and then rewind. push() is still there and working, but I'm not sure if it should still exist.


Why is that necessary?

Kronuz commented May 9, 2013

For one thing, although this is arguable, it's cleaner to have a rewind method in a lexer. For the more practical answer, the django <-> jinja2 compatibility layer I'm working on, and which I will soon make available, needs rewind for the template tags part, to compile Extensions out of simple_tag, inclusion_tag and other template-sugar-like helper function decorators that work both in jinja2 and django.
About that, by the way, I now have the exact same unmodified (really big) templates codebase working in both jinja2 and django (with my compatibility layer) and I can easily measure the benefits of using jinja2 vs django in the exact same page by swapping engines: 4697ms (django) vs 468ms (jinja2)


Closing this for the time being until a patch lands that depends on it.

@mitsuhiko mitsuhiko closed this May 19, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment