Caching does not belong in the app

[gma]

It made sense (pragmatically) to put file based caching in Nesta when it was in it's infancy, before the sinatra-cache gem really worked properly, and before Heroku was a viable deployment option. Now it just seems bizarre that caching behaviour should be baked into the app.

Would it be better to use Rack::Cache? Here's a patch from Gerhard:

http://github.com/gerhard/nesta/commit/727d9c0a089b430e07dd8b2525c71bea6d876315

File based caching is a great option for people deploying with Nginx, but the current behaviour seems to cause confusion. This comment was posted to a blog running Nesta:

I find it odd that Nesta does this cachine but the official documentation makes no mention of needing rewrite rules or anything. It gave me the impression that the app itself handled the work with regards to whether or not to serve the static html or not.

That said, anyone seen any nginx rewrite examples for this around?

I think it's time to switch the file based caching from lib/cache.rb to sinatra-cache.

[tuupola]

AFAIK with Rack::Cache the requests still go to Sinatra. For maximum efficiency funky caching (Nginx / Apache servers cached static files with rewrite rules) is still the fastest option.

I ended up writing my own Rack middleware for funky caching and have been using that with Nesta since. You can enable it with two lines of code.

require "rack/funky-cache"
use Rack::FunkyCache, :directory => Nesta::Configuration.cache_dir

Source at: http://github.com/tuupola/rack-funky-cache

[gma]

Excellent, thanks. I'll take a look at that.

[tuupola]

Forgot to mention you of course need to setup rewrite rules too. Mine are something like:

# Rewrite index to check for static.
RewriteCond  %{DOCUMENT_ROOT}/cache/index.html -f
RewriteRule ^$ /cache/index.html [L,QSA]

# Rewrite to check for cached page from cached folder.
RewriteCond %{REQUEST_METHOD} ^GET$
RewriteCond %{DOCUMENT_ROOT}/cache/%{REQUEST_URI}.html -f
RewriteRule ^(.*)$ /cache/$1.html [L,QSA]

# Main URL rewriting.
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-l
RewriteRule (.*) http://127.0.0.1:4567%{REQUEST_URI} [L,P,QSA]

[gma]

Thanks for that. I may stick it in some docs and attribute it to you if that's okay (I'm in no position to test it though).

In the interests of collecting all the caching thoughts in one place, there's a good discussion in the comments on Kyle Burckhard's site:

http://www.vorpalcode.com/nesta-on-heroku

[gma]

One of the reasons I held off on updating the caching code for so long was because it wasn't clear to me how best to implement it, and I've had some great (but different) suggestions from plenty of people about how to achieve it.

However, I'll be working on a plugin soon that just won't work without you being able to set the max-age header independently for different pages, so I'll be adding some optional Max age metadata that will be used to set the Cache-Control header.

Here's the story: https://github.com/gma/nesta/issues#issue/29

I'm not sure if I'll do anything with etags/last_modified or not yet; it'd be nice, but until I come up with an API/config file format that is flexible enough and clean enough, I'll hold off on that part.

In the meantime I'm closing this ticket, as I now have a plan. :-)

[datenimperator]

Funny to read this discussion here. After just one day working with nesta I was wondering why you'd put in your own caching but not the Sinatra module. I was just about to provide a patch :-)

To my understanding, caching belongs not to the app but to the servers in front of it. The app should provide meaningful meta information (using http headers, that is) to allow proper functionality. If frontend caches decide to let the request fall through to the app again, it might be for a reason and the app does well in general by responding with a fresh, non-cached result.