Permalink
Browse files

Make collectstatic for cached static files more "atomic".

The commit moves all cache set to the end of the post-processing phase
of the CachedStaticFilesStorage class. We do that by keeping a
dictionary of what needs to be stored to the cache and store that after
all static files have been post-processed. Additionally, existing cache
keys are never deleted; instead, we rely on the cache expiration to
remove not-recently-used keys.

This is convenient for deployments. Ususally, one first does a
collectstatic and then restarts his application servers. But since the
cache is (normally) shared with the currently running application
processes, any change to the static files will be immediately used for
existing requests. This results in having a previous codebase serving
newer static files, which might lead to issues.

The commit moves all cache handling to the end, so that the above window
of serving wrong versions of the static files is minimized. This has the
added benefit, that one can safely stop the collectstatic command in the
middle of the execution, without any impact on the existing application
processes (the old files are always there, anyway).

Signed-off-by: Jannis Leidel <jannis@leidel.info>
  • Loading branch information...
1 parent 477aff5 commit 223cbbb8f2964fe7cd2cc96d1603f1ffd32022fc Apostolos Bessas committed with jezdez May 5, 2012
Showing with 5 additions and 3 deletions.
  1. +5 −3 staticfiles/storage.py
View
@@ -234,8 +234,8 @@ def post_process(self, paths, dry_run=False, **options):
if dry_run:
return
- # delete cache of all handled paths
- self.cache.delete_many([self.cache_key(path) for path in paths])
+ # where to store the new paths
+ hashed_paths = {}
# build a list of adjustable files
matches = lambda path: matches_patterns(path, self._patterns.keys())
@@ -284,9 +284,11 @@ def post_process(self, paths, dry_run=False, **options):
hashed_name = force_unicode(saved_name.replace('\\', '/'))
# and then set the cache accordingly
- self.cache.set(self.cache_key(name), hashed_name)
+ hashed_paths[self.cache_key(name)] = hashed_name
yield name, hashed_name, processed
+ self.cache.set_many(hashed_paths)
+
class CachedStaticFilesStorage(CachedFilesMixin, StaticFilesStorage):
"""

0 comments on commit 223cbbb

Please sign in to comment.