Concurrency bug in LRUCache #8

Closed
jvs opened this Issue Nov 24, 2010 · 2 comments

Projects

None yet

2 participants

@jvs
jvs commented Nov 24, 2010

I think the LRUCache class in Jinja2 has a bug. The problem is that the "getitem" method may mutate the cache, but that method does not acquire the write lock, "_wlock".

For example, here's one scenario:

  • Thread A calls "getitem" with the key "K".
  • "K" is not the last element of the queue, so Thread A removes it.
  • Thread A is suspended before it appends "K" to the queue.
  • Thread B calls "setitem" with the key "K".
  • Thread B finds "K" in the "mapping" dictionary.
  • Thread B tries to remove "K" from the queue.
  • The queue raises an exception, because it does not contain "K".

In my project I've seen the "setitem" method raise a KeyError. I assume that there's a similar execution scenario behind those exceptions, but I could be wrong.

In any case, I think "getitem" clearly needs to acquire "_wlock" before it removes and appends keys.

@jvs
jvs commented Nov 24, 2010

[Markdown messed up my report. Let me try again.]

I think the LRUCache class in Jinja2 has a bug. The problem is that the "getitem" method may mutate the cache, but that method does not acquire the write lock, "_wlock".

For example, here's one scenario:

  • Thread A calls "getitem" with the key "K".
  • "K" is not the last element of the queue, so Thread A removes it.
  • Thread A is suspended before it appends "K" to the queue.
  • Thread B calls "setitem" with the key "K".
  • Thread B finds "K" in the "mapping" dictionary.
  • Thread B tries to remove "K" from the queue.
  • The queue raises an exception, because it does not contain "K".

In my project I've seen the "setitem" method raise a KeyError. I assume that there's a similar execution scenario behind those exceptions, but I could be wrong.

In any case, I think "getitem" clearly needs to acquire "_wlock" before it removes and appends keys.

@tonytargonski

To add to this 2 year old ticket... I have also observed the same exception in my system.

[Tue Feb 12 19:48:16 2013] [error]   File "/local/python_envs/www/local/lib/python2.7/site-packages/jinja2/utils.py", line 473, in __setitem__
[Tue Feb 12 19:48:16 2013] [error]     del self._mapping[self._popleft()]
[Tue Feb 12 19:48:16 2013] [error] KeyError: '_breadcrumbs.html
@mitsuhiko mitsuhiko closed this in d4e5438 Apr 12, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment