Permalink
469 lines (328 sloc) 15.6 KB

Varnish Configuration

Below you will find detailed Varnish configuration recommendations for the features provided by this library. The configuration is provided for Varnish 3, 4 and 5.

Basic Varnish Configuration

To invalidate cached objects in Varnish, begin by adding an ACL (for Varnish 3 see ACL for Varnish 3) to your Varnish configuration. This ACL determines which IPs are allowed to issue invalidation requests. To use the provided configuration fragments, this ACL has to be named invalidators. The most simple ACL, valid for all Varnish versions from 3 onwards, looks as follows:

# /etc/varnish/your_varnish.vcl

acl invalidators {
    "localhost";
    # Add any other IP addresses that your application runs on and that you
    # want to allow invalidation requests from. For instance:
    # "192.168.1.0"/24;
}

Important

Make sure that all web servers running your application that may trigger invalidation are whitelisted here. Otherwise, lost cache invalidation requests will lead to lots of confusion.

Provided VCL Subroutines

In order to ease configuration we provide a set of VCL subroutines in the resources/config directory of FOSHttpCache. These can be included from your main Varnish configuration file, typically default.vcl. Then you need to make your vcl_* subroutines call the fos_* routines.

Tip

When including one of the provided VCL, you need to call all the defined subroutines or your configuration will not be valid.

See the respective sections below on how to configure usage of each of the provided VCLs.

Purge

Purge removes a specific URL (including query strings) in all its variants (as specified by the Vary header).

Subroutines are provided in resources/config/varnish-[version]/fos_purge.vcl. To enable this feature, add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_purge.vcl";

        sub vcl_recv {
            call fos_purge_recv;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_purge.vcl";

        sub vcl_recv {
            call fos_purge_recv;
        }

        sub vcl_hit {
            call fos_purge_hit;
        }

        sub vcl_miss {
            call fos_purge_miss;
        }

Read more on handling PURGE requests in the Varnish documentation (for Varnish 3, see purging for Varnish 3).

Refresh

Refresh fetches a page from the backend even if it would still be in the cache, resulting in an updated cache without a cache miss on the next request.

Refreshing applies only to a specific URL including the query string, but not its variants.

Subroutines are provided in resources/config/varnish-[version]/fos_refresh.vcl. To enable this feature, add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_refresh.vcl";

        sub vcl_recv {
            call fos_refresh_recv;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_refresh.vcl";

        sub vcl_recv {
            call fos_refresh_recv;
        }

Read more on forcing a refresh in the Varnish documentation (for Varnish 3, see refreshing for Varnish 3).

Ban

Banning invalidates whole groups of cached entries with regular expressions.

Subroutines are provided in resources/config/varnish-[version]/fos_ban.vcl To enable this feature, add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_ban.vcl";

        sub vcl_recv {
            call fos_ban_recv;
        }

        sub vcl_backend_response {
            call fos_ban_backend_response;
        }

        sub vcl_deliver {
            call fos_ban_deliver;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_ban.vcl";

        sub vcl_recv {
            call fos_ban_recv;
        }

        sub vcl_fetch {
            call fos_ban_fetch;
        }

        sub vcl_deliver {
            call fos_ban_deliver;
        }

This subroutine also sets the X-Url and X-Host headers on the cache object. These headers are used by the Varnish ban lurker that crawls the content to eventually throw out banned data even when it’s not requested by any client. Read more on handling BAN requests in the Varnish documentation (for Varnish 3, see banning for Varnish 3).

Tagging

Feature: :ref:`cache tagging <tags>`

For this feature you need to choose between either BAN _(default)_ and the more performant xkey based tag system.

Tag Invalidation Using xkey

Since Varnish 4.1, you can use the official xkey Varnish module for better performance with cache tags.

As explained in the Cache Invalidation chapter of the Varnish documentation:: > (..), hashtwo/xkey is much more efficient than bans because of two reasons: > 1) looking up hash keys is much more efficient than traversing ban-lists, and > 2) every time you test a ban expression, it checks every object in the cache > that is older than the ban itself.

With xkey, you can also soft purge tagged responses to allow for a grace period where invalidated (stale) content is still delivered to the client.

Minimum version of varnish-modules is v0.10.2*. It's part of Ubuntu 17.10 ("Artful") and higher, otherwise see their documentation for installing xkey on other platforms. You can technically use earlier versions of the varnish modules down to 0.9.x releases. However, v0.10.2 is the first version introducing support for purging several tags at once. If you use an older version, you need to invalidate only one tag at a time, and can't use soft purge.

To use xkey, :ref:`configure the Varnish Client for xkey <varnish_custom_tags_header>` and :ref:`the response tagger to use the xkey header <response_tagger_optional_parameters>`, and include resources/config/varnish/fos_tags_xkey.vcl in your VCL:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_tags_xkey.vcl";

        sub vcl_recv {
            call fos_tags_xkey_recv;
        }

        sub vcl_deliver {
            call fos_tags_xkey_deliver;
        }

Note that there is no xkey VCL file for Varnish version 3 because the varnish-modules are only available for Varnish 4.1 or newer.

Tag Invalidation Using BAN

If you have included fos_ban.vcl, tagging will be automatically enabled with the X-Cache-Tags header for both marking the tags on the response and for the invalidation request to tell what tags to invalidate.

If you use a different name for :doc:`response tagging <response-tagging>` than the default X-Cache-Tags or a different name for specifying which tags to invalidate in your :ref:`cache invalidator configuration <varnish_custom_tags_header>` you have to write your own VCL code for tag invalidation. Your custom VCL will look like this:

.. configuration-block::

    .. literalinclude:: ../resources/config/varnish/fos_ban.vcl
        :language: varnish
        :emphasize-lines: 17-23,50-51
        :linenos:

    .. literalinclude:: ../resources/config/varnish-3/fos_ban.vcl
        :language: varnish3
        :emphasize-lines: 17-23,50-51
        :linenos:

Hint

The line you need to adjust from the code above is line 21. The left side is the header used to tag the response, the right side is the header used when sending invalidation requests. If you change one or the other header name, make sure to adjust the configuration accordingly.

User Context

Feature: :doc:`user context hashing <user-context>`

The fos_user_context.vcl needs the user_context_hash_url subroutine that sets the URL to do the hash lookup. The default URL is /_fos_user_context_hash and you can simply include resources/config/varnish-[version]/fos_user_context_url.vcl in your configuration to provide this. If you need a different URL, write your own user_context_hash_url subroutine instead.

Tip

The provided VCL to fetch the user hash restarts GET/HEAD requests. It would be more efficient to do the hash lookup request with curl, using the curl Varnish plugin. If you can enable curl support, the recommended way is to implement your own VCL to do a curl request for the hash lookup instead of using the VCL provided here.

Also note that restarting a GET request leads to Varnish discarding the body of the request. If you have some special case where you have GET requests with a body, use curl.

To enable this feature, add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_user_context.vcl";
        include "path-to-config/varnish/fos_user_context_url.vcl";

        sub vcl_recv {
            call fos_user_context_recv;
        }

        sub vcl_backend_response {
            call fos_user_context_backend_response;
        }

        sub vcl_deliver {
            call fos_user_context_deliver;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_user_context.vcl";
        include "path-to-config/varnish/fos_user_context_url.vcl";

        sub vcl_recv {
            call fos_user_context_recv;
        }

        sub vcl_fetch {
            call fos_user_context_fetch;
        }

        sub vcl_deliver {
            call fos_user_context_deliver;
        }

Caching User Specific Content

By default, Varnish does not check for cached data as soon as the request has a Cookie or Authorization header, as per the builtin VCL (for Varnish 3, see default VCL). For the user context, you make Varnish cache even when there are credentials present.

You need to be very careful when doing this: Your application is responsible for properly specifying what may or may not be shared. If a content only depends on the hash, Vary on the header containing the hash and set a Cache-Control header to make Varnish cache the request. If the response is individual however, you need to Vary on the Cookie and/or Authorization header and probably want to send a header like Cache-Control: s-maxage=0 to prevent Varnish from caching.

Your backend application needs to respond to the application/vnd.fos.user-context-hash request with :ref:`a proper user hash <return context hash>`.

Tip

The provided VCL assumes that you want the context hash to be cached, so we set the req.url to a fixed URL. Otherwise Varnish would cache every hash lookup separately.

However, if you have a :ref:`paywall scenario <paywall_usage>`, you need to leave the original URL unchanged. For that case, you would need to write your own VCL.

Cleaning the Cookie Header

In the examples above, an unaltered Cookie header is passed to the backend to use for determining the user context hash. However, cookies as they are sent by a browser are unreliable. For instance, when using Google Analytics, cookie values are different for each request. Because of this, the hash request would not be cached, but multiple hashes would be generated for one and the same user.

To make the hash request cacheable, you must extract a stable user session id before calling fos_user_context_recv. You can do this as explained in the Varnish documentation:

Note

If your application’s user authentication is based on a cookie other than PHPSESSID, change PHPSESSID to your cookie name.

Custom TTL

Subroutines are provided in resources/config/varnish-[version]/fos_custom_ttl.vcl. Add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_custom_ttl.vcl";

        sub vcl_backend_response {
            call fos_custom_ttl_backend_response;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_custom_ttl.vcl";

        sub vcl_fetch {
            call fos_custom_ttl_fetch;
        }

The custom TTL header is removed before sending the response to the client.

Note

If you are using Varnish 3, this feature is using inline C. Inline C is enabled for Varnish 3 by default. Check for the vcc_allow_inline_c setting. If you are using Varnish 4 or newer, you are using the varnish/fos_custom_ttl.vcl which uses a vmod function instead of inline C.

Debugging

Configure your Varnish to set a custom header (X-Cache) that shows whether a cache hit or miss occurred. This header will only be set if your application sends an X-Cache-Debug header:

Subroutines are provided in fos_debug.vcl.

To enable this feature, add the following to your_varnish.vcl:

.. configuration-block::

    .. code-block:: varnish

        include "path-to-config/varnish/fos_debug.vcl";

        sub vcl_deliver {
            call fos_debug_deliver;
        }

    .. code-block:: varnish3

        include "path-to-config/varnish-3/fos_debug.vcl";

        sub vcl_deliver {
            call fos_debug_deliver;
        }