Some stanzas (with roster for example) can contain huge amount of xml, which is all parsed into xml tree even if use_raw=True. Because of this punjab can consume large amount of RAM (possibly because of python memory fragmentation http://www.gossamer-threads.com/lists/python/python/1162114) and CPU (to create domish.Element() instances).
ShallowExpatElementStream overrides _onStartElement() and _onEndElement() methods of regular parser to avoid building whole tree, which is done by default in domish.py:ExpatElementStream.
Don't build whole xml tree for stanza payload
Don't use no longer supported --use-mirrors
Use twisted from github instead of svn
Workaround for failing trial runs for latest twisted version
Thanks for this, I will look at it as soon as I can.
This looks good. The changes to .travis.yml are not related but that is ok. Just need the small change to handle the possibility of not having the expat dependency.
elementStream returns an ExpatElementStream or an SuxElementStream depending if expat is available or not. If someone is running punjab without that dependency it will fail.
If you could mimic this function, I think that would be ok.
Thanks for noticing this. I've added elementStream() func, almost the same as in domish.py. I can also make ShallowSaxElementStream, but not sure if it is necessary.
Fallback to Sux if Expat is not available