New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache tag support to default Varnish config #397

Closed
geerlingguy opened this Issue Jan 26, 2016 · 15 comments

Comments

4 participants
@geerlingguy
Copy link
Owner

geerlingguy commented Jan 26, 2016

Documentation:

As I like to use Drupal VM as a good example of best practices for infrastructure configuration (e.g. the switch to PHP-FPM + Apache as a default), I'd also like to support D8 cache tags with Varnish out of the box, partly because I'm starting to build D8 sites that will rely on this functionality, and partly just because.

Also, for one good real-world use case of D8 + cache tags + Fastly in the wild, see: http://www.md-systems.ch/en/blog/2015-10/successful-relaunch-of-letemps-ch

@achton

This comment has been minimized.

Copy link

achton commented Feb 8, 2016

Indeed. We are also contemplating how to implement this in our hosting infrastructure in the best & simplest manner, and I had also looked at FOSHttpCache for pointers. Monitoring this closely, and will share if we come up with anything useful.

@geerlingguy geerlingguy added planned and removed enhancement labels Feb 21, 2016

@achton

This comment has been minimized.

Copy link

achton commented Feb 24, 2016

@geerlingguy what is your plan here, if I may ask? Have you considered whether Purge module can help (the architecture is explained in the 3.0-beta1 release notes), or are you going the way of wrapping around FosHttpCache? Or something else entirely?

Are you going to make use of the http.response.debug_cacheability_headers: true setting (now only recommended for development) to expose the X-Drupal-Cache-Tags header? Or how else do you intend on exposing the tags for Varnish?

And finally, which Varnish implementation to use? The Surrogate keys module for Varnish has been open sourced and may do the trick, but there is also the VCL snippet from FosHTTPCache which seems simpler.

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Feb 25, 2016

On the Drupal side, I'm thinking you'll want to use Purge to make sure cache tags are purged when appropriate.

On the Varnish side, I want to just make a VCL change. Don't want to add any additional modules or require any extra software/configuration. It should work OOTB with Varnish 4.x, I think. Just need to do the work to change over the default VCL that ships with Drupal VM.

Probably worthy of a blog post as well, to show people how they can start using cache tags with Varnish in Drupal 8 easily...

@geerlingguy geerlingguy added this to the 3.0.0 milestone Mar 5, 2016

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 19, 2016

Reading up as much as I can on various options and caveats with Bans, I'm also hoping to find a definitive answer to Wim's question from 2014: https://www.smashingmagazine.com/2014/04/cache-invalidation-strategies-with-varnish-cache/#comment-453091

Also, see:

And whatever findings I come up with, we should probably document things better on the 'semi-official' Varnish and Drupal wiki page on Varnish's site: https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 21, 2016

Some notes in getting Cache Tags working with purging:

  1. Set http.response.debug_cacheability_headers: true inside services.yml (but see this comment/issue for more info on sending the right tags).
  2. Updates to the drupalvm.vcl.j2 file (will be posted in a PR here soon) based roughly on those in the FOSHTTPCache library.

Current simple logic inside vcl_recv is:

    # Only allow BAN requests from IP addresses in the 'purge' ACL.
    if (req.method == "BAN") {
        # Same ACL check as above:
        if (!client.ip ~ purge) {
            return (synth(403, "Not allowed."));
        }

        # Logic for the ban, using the X-Drupal-Cache-Tags header.
        if (req.http.X-Drupal-Cache-Tags) {
            ban("obj.http.X-Drupal-Cache-Tags ~ " + req.http.X-Drupal-Cache-Tags);
        }
        else {
            return (synth(403, "X-Drupal-Cache-Tags header missing."));
        }

        # Throw a synthetic page so the request won't go to the backend.
        return (synth(200, "Ban added."));
    }

Using obj rather than any kind of req matching (this can require a little setup inside vcl_backend_response and cleanup for the client response sent inside vcl_deliver) allows the ban lurker to clean up the bans every so often, and I've confirmed that banning works (causing the next request for a previously-cached page with that tag present to get a X-Varnish-Cache: MISS, then the one thereafter to get another X-Varnish-Cache: HIT), and also that the lurker cleans up the old bans based on the obj.

Testing bans

To test bans without installing anything extra within Drupal, you can send BAN requests via curl.

# Send a BAN to Varnish.
$ curl -X BAN http://127.0.0.1:81/ -H "X-Drupal-Cache-Tags: block_view"
$ curl -X BAN http://127.0.0.1:81/ -H "X-Drupal-Cache-Tags: node:1"

# Check the current list of Varnish bans.
$ varnishadm
varnish> ban.list
200        
Present bans:
1458593353.734311     6    obj.http.X-Drupal-Cache-Tags ~ block_view

# Check the current parameters.
varnish> param.show
...
ban_dups                   on [bool] (default)
ban_lurker_age             60.000 [seconds] (default)
ban_lurker_batch           1000 (default)
ban_lurker_sleep           0.010 [seconds] (default)
...

(Will be updated as I go.)

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 21, 2016

See above PR.

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 21, 2016

To set up purging support within Drupal:

  1. Install purge_purger_http, which depends on purge: drush en -y purge_purger_http.
  2. Enable Purge, Purge Drush, Purge UI, Core tags queuer, and Generic HTTP Purger.
  3. Visit the Purge UI admin interface (admin/config/development/performance/purge) and click "Add purger".
  4. Select HTTP Bundled Purger, and hit a Fatal error...
  5. Post issue about said error with a fix: https://www.drupal.org/node/2691849#comment-10993987
  6. Do it again, configure the Bundled Pager with the following:
    • Name: Varnish Cache Tag Purger
    • Request: localhost, port 81, path /, request method BAN
    • Headers: X-Drupal-Cache-Tags with token [invalidations:separated_comma]
  7. Update /about page (node 1), run Drush queue worker with drush @drupalvm.drupalvm.dev p-queue-work
  8. Get 405 error :(
  9. Post issue about the bug with a fix: https://www.drupal.org/node/2691913
  10. Do the same thing as number 7 above, and SUCCESS!

geerlingguy added a commit that referenced this issue Mar 22, 2016

@nielsvm

This comment has been minimized.

Copy link

nielsvm commented Mar 22, 2016

I just pushed 8.x-1.0-beta2 at https://www.drupal.org/node/2692065 (still 403).

@nielsvm

This comment has been minimized.

Copy link

nielsvm commented Mar 22, 2016

Set http.response.debug_cacheability_headers: true inside services.yml (but see this comment/issue for more info on sending the right tags).

This shouldn't be necessary as purge takes care of this, though I need to change its implementation (but resulting behavior will stay the same).

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 22, 2016

@nielsvm - Thanks for the clarification; doing some digging, I did notice purge.services.yml sets the default:

parameters:

  # Enable Drupal's X-Drupal-Cache-Tags response header. This HTTP response
  # header describes what got rendered onto every page with descriptive text
  # keys uniquely identifying every rendered object. This then enables external
  # caching platforms - exactly what this module is for - to leverage this.
  http.response.debug_cacheability_headers: true

...however, if someone were to copy core's default.services.yml to services.yml in their site folder, then the default defined there (http.response.debug_cacheability_headers: false) will override the Purge module's default. So we definitely need to make it clear that if the headers aren't being sent, check and make sure you're not disabling the headers.

Or another option would be to do like the Fastly module and build in your own header (maybe even with a customizable name, defaulting to X-Cache-Tags?). Anyways, it looks like everything's working great now, with the latest beta releases of both modules; and no need for the Varnish module at all at this point!

geerlingguy added a commit that referenced this issue Mar 22, 2016

Merge pull request #525 from geerlingguy/varnish-cache-tags
Issue #397: Add basic Varnish cache tag support with BANs.
@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 22, 2016

Working on a blog post with documentation for usage now.

@geerlingguy

This comment has been minimized.

@nielsvm

This comment has been minimized.

Copy link

nielsvm commented Mar 22, 2016

@geerlingguy thank you SO much for that article, very, very happy reading it and sharing it where I can. By no means is the purge_purger_http intended to be the Varnish solution but in the meanwhile, this is all we got. I've been too busy to reach out to the varnish module maintainer for integration, so this module is the fallback solution.

In any case its safe to say that we're getting closer achieving our goal of perfect external cache invalidation on Drupal!

@wimleers

This comment has been minimized.

Copy link

wimleers commented Mar 23, 2016

I'm also hoping to find a definitive answer to Wim's question from 2014

I had cross-posted that comment to http://info.varnish-software.com/blog/advanced-cache-invalidation-strategies, but then they migrated their Drupal 7 site to a re-architected Drupal 7 site … and lost all comments in the process! Whoever did that migration, shame on you.

I have a cached copy, but sadly one that is from before my comment and Per Buer's reply: https://pinboard.in/cached/a31c4630dd2d/

I pinged @perbu on Twitter to ask him to restore those comments: https://twitter.com/wimleers/status/712573035132428288

But his answer was basically this (paraphrased from memory):

Varnish can easily handle thousands of invalidations per second.

In other words: only at enormous scale — i.e. an invalidation frequency of >1000Hz — will you start to see problems. And then there is still hashtwo, which was open sourced for Varnish 4: libvmod-xkey — see https://www.varnish-cache.org/docs/trunk/whats-new/changes.html:

libvmod-xkey: Secondary hash keys for cache objects, based on the hashtwo vmod written by Varnish Software. Allows for arbitrary grouping of objects to be purged in one go, avoiding use of ban invalidation. Also known as Cache Keys or Surrogate Key support.

@geerlingguy

This comment has been minimized.

Copy link
Owner

geerlingguy commented Mar 23, 2016

@wimleers - Thanks for the added context—that was my main worry, but in some simple testing I've been doing over the past few days, even with a few thousand bans in short succession, Varnish load barely showed any difference, and the ban lurker quickly cleaned out old requests.

I can see this technique scaling to most sites' traffic/content update levels quite easily. If you're a forum with hundreds of thousands of content updates per hour... then you'll probably be running into other issues before you have to worry about the lurker and optimize for libvmod-xkey. Though I'm going to try doing some testing on it as well, because it would be helpful to use in some situations!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment