Jinja2 needs a timeout capability #161

Closed
ntrrgc opened this Issue Nov 14, 2012 · 9 comments

Projects

None yet

3 participants

@ntrrgc
ntrrgc commented Nov 14, 2012

For systems that let users write their own code, being able to set an execution timeout for the template processor is sometimes a must.

Otherwise it is very easy for users to perform a DoS, even if it's an accident. Furthermore, if the system allows untrusted people to run arbitrary (managed) Jinja code, this lack is critical.

@soulseekah

Could you please describe your particular setup/environment? How are users limited? What are users permitted to do? How does Jinja let them to do things that they are not permitted to do in your environment? How are users limited from running arbitrary code in Python directly, do they only have access to Jinja templates only?

Sorry for so many questions, but it seems that "systems that let users write their own code" are generally vulnerable; do you have any sort of system in place to limit execution time of any Python piece code? Why would rendering timeout in Jinja be the solution to your specific issue? It seems to be more of a global requirement. For example, WSGI containers usually have global timeouts per worker (like the harakiri setting for uWSGI), mod_wsgi surely has something similar.

The more information (including examples), the better. Thanks.

@ntrrgc
ntrrgc commented Nov 14, 2012

Well, is not mine, but friend's system which has this problem:

My friend has an XMPP chat bot. The bot enters into a series of chat rooms, and stays there waiting for messages.

This bot has a popular module, usercommands. This module allows users writing and installing 'commands' in the bot. This commands are small programs written is Jinja, which have limited access to a set of variables (these variables store the nick of the user who runs the command, the arguments the command is invoked with, the list of currently connected users in the room, the name of the room, and little more).

With this, users can write small safe programs which do things like throwing virtual dices, attacking other users with random virtual weapons, storing funny quotes, and so. They're mostly for fun (although they can be used for many things, like storing frequently answered questions and answers, like bots of many programming chat rooms do).

Users are allowed to write and run commands once they are known, regular users of the chat room (they are granted by moderators of the room, which are known by the bot).

Although funny, and mostly secure (the commands run in an isolated environment, as Jinja2 with controlled context variables is), this system has a flaw because we can't set a timeout to commands nor stop them. If an user writes and runs a buggy command which enters into a quasi-infinite loop*, Jinja2 does not provide a method to stop the processing of that template. Once this happens, the bot keeps working, cause it's multi-threaded, but it keeps hogging 100% CPU, at least on a core.

*Last time I put it into that state by telling it to calculate China population factorial.

@soulseekah

Interesting, thanks for sharing. Jinja2 was never ever meant for this, it's a templating engine and should never ever have to do any considerable amount of work/processing/logic other than loops and conditional branches. But that aside, the XMPP bot has to have timeouts when invoking any long-running functions, is it written in Python? If it is look into http://stackoverflow.com/questions/2281850/timeout-function-if-it-takes-too-long-to-finish which would be a fantastic way to limit Jinja rendering time without imposing any additional responsibility on Jinja itself.

@ntrrgc
ntrrgc commented Nov 15, 2012

Yes, the bot is entirely written in python.

The approach of the first SO answer won't work, because SIGALRM is not thread-safe, and our bot is multi-threaded.

The other guy tells us to set a timer thread to send 'an interrupt' to the worker if it's still running, and points us to the threading API documentation.

Looking at those docs, I only find a function for sending an interrupt (^C) to the main thread. May I being missing something?

@soulseekah

It's hard to say without looking at your actual code. There are many different way to deal with threading and killing long-running threads, but there's many different ways to manage threads as well, so not sure what to suggest. Easiest may be setting the timeout argument to join http://docs.python.org/2/library/threading.html#threading.Thread.join perhaps.

The second SO answer refers to http://docs.python.org/2/library/threading.html#event-objects

http://code.activestate.com/recipes/473878-timeout-function-using-threading/
http://www.unixunion.org/index.php/component/content/article/5-general/4-python-threading-timers
http://code.activestate.com/recipes/576780-timeout-for-nearly-any-callable/

You can see how complex some of the solutions are, and you wouldn't want Jinja to even start thinking about managing its own rendering timeouts, it's not meant for this.

@soulseekah

By the way, this is where the generated Jinja code gets executed https://github.com/mitsuhiko/jinja2/blob/2.6/jinja2/environment.py#L834 exec code in namespace is where it all happens.

More ideas:

http://www.velocityreviews.com/forums/t322179-exec-code-with-timeout.html
switch to multiprocessing perhaps
http://code.activestate.com/recipes/496746-restricted-safe-eval/ (look at exec_timed)

@ntrrgc
ntrrgc commented Nov 15, 2012

It seems the only reliable solutions need polling.

@soulseekah

One would not expect Jinja2 to have "this" as part of its core, right?

@mitsuhiko
Member

We can't do that nicely, indeed would require moving that into a separate process and timing that out. I think that's out of the scope of Jinja.

@mitsuhiko mitsuhiko closed this May 20, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment