Build management command

[dfunckt]

Right now django_compressor can get very slow when having to package many/large files -- i've seen more than 2.5 secs added in page response time, just to figure out that no file was modified!

A ./manage.py compress_media command that would build the assets would be very handy. Coupled with a DISABLE_AUTO_COMPRESS setting would allow for manual builds providing the best of both worlds -- compressed/packaged assets + no processing overhead. django-compressor does this and i find it is very useful.

Thanks!

Akis

[jezdez]

I'm not sure how that should work given the fact the media files are reference in the templates and not outside in the settings or similar. In other words, can you think of a way to walk all available templates, find the compress template nodes to trigger the compressing?

[dfunckt]

I tried to figure out a way to do it before initially posting the issue, however i couldn't find any solution that wouldn't require several design decisions.

However, one clever, if inelegant, way to do this would be to somehow flag whether the compress tag should skip processing and just replace the files with the cached file, or decide to re-check modification times and process.

A possible location to store state is a file inside the cache dir, where if its contents validate, the template tag would skip processing. If it doesn't validate or doesn't exist, it would process as normal. This way, all the management command had to do would be to delete this file, so the next request would force the tag to re-process.

Hopefully this is making sense.

Akis

[jezdez]

Hm, just out of curiosity, have you tried the COMPRESS_MTIME_DELAY setting? When specified it'll check for file changes only every those seconds. It requires a working setup of Django's caching.

[dfunckt]

Yes, i am aware of this setting, however i haven't got to use it.

There is one issue with it though. The interval has to be small -- having to be smaller than the processing timeout setting -- so compilation will happen at least once a minute -- assuming a sane timeout. And it wouldn't make sense to be allowed to be larger -- say once per day -- as that would mean that you'd have to wait for the right time for your newly updated assets to re-compile.

So, at least for a production site, that could be updated from several times a day to eg. once per month depending on the project, a way is needed to avoid the additional overhead from the files mtime check. A management command makes much sense -- and it could even be automated via fabric etc.

Akis

[gvangool]

I've been thinking about this problem. On our site compressor added 5 seconds to response times, so we had to disable it.
I've just created a mgmt command that will load all mtimes to the cache. This way, - I hope - we can set COMPRESS_MTIME_DELAY to something like 31 days.

[jezdez]

Nice command, going to check it out later on a staging server..

[jezdez]

I've merged the command with a few modifications (such as more generic ignore options). Thanks!