Permalink
Browse files

query_cache: Limit prune deletion size for safety.

If prunings fail, the number of items in a fast-moving cached query
could grow rather large.  This prevents a bad case where the pruning
ends up way too big and fails itself, causing the query to snowball in
size.
  • Loading branch information...
1 parent 8c62ace commit cf68046674d339875d235b1162d42b31f7e481f2 @spladug spladug committed Oct 10, 2012
Showing with 7 additions and 0 deletions.
  1. +7 −0 r2/r2/models/query_cache.py
@@ -195,6 +195,13 @@ def _prune(self, mutator):
extraneous_ids = [t[0] for t in self.data[MAX_CACHED_ITEMS:]]
if extraneous_ids:
+ # if something has gone wrong with previous prunings, there may be
+ # a lot of extraneous items. we'll limit this pruning to the
+ # oldest N items to avoid a dangerously large operation.
+ # N = the average number of items to prune (doubled for safety)
+ prune_size = int(MAX_CACHED_ITEMS * PRUNE_CHANCE) * 2
+ extraneous_ids = extraneous_ids[-prune_size:]
+
self.model.remove_if_unchanged(mutator, self.key,
extraneous_ids, self.timestamps)

0 comments on commit cf68046

Please sign in to comment.