Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP


Stream operation for pystache #108

gonvaled opened this Issue · 5 comments

2 participants


This is what I usually do to render a file with pystache:

txt = open(filename, 'r').read()
print pystache.render(txt, context).encode('utf-8')
("context" is my mapping)

But now I have to render very big templates, which causes memory problems. It would be great if pystache could operate on streams.


Thanks for the report. For future reference, using memory mapped files might be another possible approach to dealing with large files, but I'm not sure.


That is a good idea. For the input data this would probably work, but (and I am talking here without knowing the details of the pystache implementation) I guess this would not completely solve the issue, since the output data would still be in memory


And it seems not even input mmap is supported:

import mmap
import pystache
tmp_file = '/tmp/tmp_file'
template_text = 'Hello {{name}}'
d = {'name' : 'Steve'}
f = open(tmp_file, 'w')
f = open(tmp_file, 'r+b')
mymmap = mmap.mmap(f.fileno(), 0)
print pystache.render(mymmap, d)
» python 
Traceback (most recent call last):
  File "", line 11, in 
    print pystache.render(mymmap, d)
  File "/home/gonvaled/.virtualenvs/python2.7.2-wavilon1/lib/python2.7/site-packages/pystache/", line 8, in render
    return Template(template, context).render()
  File "/home/gonvaled/.virtualenvs/python2.7.2-wavilon1/lib/python2.7/site-packages/pystache/", line 176, in render
    result = self._render_tags(template)
  File "/home/gonvaled/.virtualenvs/python2.7.2-wavilon1/lib/python2.7/site-packages/pystache/", line 119, in _render_tags
    template = template.replace(tag, replacement)
AttributeError: 'mmap.mmap' object has no attribute 'replace'

Just to clarify, I didn't mean to suggest that mmap would work without making additional changes. :) Work would need to be done there, too (but probably not as much). And yes, you are right. If support for mmap were added to the API, we would not want those methods to return the output as a string, etc.


Could you describe what your large files "look" like? For example, are they deeply nested with sections, is it long strings of plain text in between a few scattered tags, or is it equally dense with both plain text and tags? Also, are there section tags that contain huge strings, etc?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.