PageCaching.txt

---+!! Page Caching

%TOC%

By default, Foswiki pages are delivered to the browser using HTML, which is
regenerated from the data stored in Foswiki each time the  page is requested.
Page caching is the process of storing (caching) this HTML on the server, as
and when it is generated, so that the next time someone asks for the same page
it can be delivered from the cache instead of having to be re-rendered from
scratch. Page caching can significantly improve the performance of a Foswiki
site, especially where most requests are views (this is normal on most sites).
Since version 1.1, Foswiki has built-in page caching.

For most Foswiki installations you can enable page caching by simply selecting
the ={Cache}{Enabled}= option in the Tuning section of =configure=. However to
get the most out of the cache, you need to understand what it is doing, and may
want to tune the cache options for your site.

When Foswiki generates a page, it may refer to a lot of content stored in the
Foswiki database. For example, a topic 'MyTopic' may be rendered differently
depending on a setting in !WebPreferences, which is itself a topic. If
!WebPreferences were to be edited, then the cache of !MyTopic would have to be
regenerated, because the setting it depends on might have been changed during
the edit. For an average page there are a large number of these _dependencies_
that have to be tracked. This tracking has to be fast, so the cache uses a
database engine to record the dependencies and save the rendered pages.

---++ Using the Cache
By its very nature, a cache should be invisible to the end user: editing and viewing
a page should always deliver up-to-date content. To do so Foswiki's page cache
tracks the ingredients used to render the page and keeps track of any change of them.

For example, a cached topic that [[VarINCLUDE][includes]] another topic will
be refreshed automatically whenever the included topic changes.

Foswiki's page cache performs a so called _deep dependency tracking_ of every
page it caches and stores this dependency graph into its [[#metadatadb][meta-data DB]].
Based on this information, cache entries of topics are invalidated automatically.

While works out for most normal wiki use cases, this kind of cache maintenance
is not able to fully track all required refreshes.

There are some things users may have to know to get the best out of the
Foswiki cache.

---+++ Topics that always change
Some other topics change automatically whenever _any_ topic in the same web is
saved. These are usually topics that are generated by searches over the
content. You can list the names of these topics a =WEBDEPENDENCIES= web
preference setting. This is a comma-separated list of topics names.

<a name="dirtyarea"></a>
---+++ Dirty Areas
Sometimes caching complete pages is too coarse-grained. There may be parts of a
page that change frequently, while the rest of the same page never change. In
this case the author of the topic can tell the cache not to save certain parts
of it, called _dirty areas_. Dirty areas are marked in the topic using the
=&lt;dirtyarea&gt;...&lt;/dirtyarea&gt;= tags. Foswiki markup within a dirty
area is stored unrendered in the cache, and only rendered when the topic is
served from the cache. For example,

<verbatim class="tml">
This page was cached at %SERVERTIME%.

<dirtyarea> The current time of your request is %SERVERTIME%. </dirtyarea>
</verbatim>

ensures that the cache will never store the SERVERTIME expression inside the =&lt;dirtyarea> section=, forcing it to be
re-computed every time the page is served.

---+++ Refreshing the cache
Sometimes it is necessary to force a cache refresh from the client. To support
this Foswiki provides the =refresh= parameter, which works with all scripts
that produce cacheable output. You may force the current topic to be recomputed
by specifying =refresh=on= or =refresh=cache=. 

<verbatim class="tml">
<a href="%SCRIPTURLPATH{"view"}%/%WEB%/%TOPIC%?refresh=cache">Refresh this page.</a>
</verbatim>

The complete cache for all topics can be cleared using =refresh=all=. 

---+++ Caching SEARCH results

When you enable the Foswiki page cache, all SEARCH results will automatically
be cached. As a consequence a SEARCH on a page will not be performed again as long as this
page is cached. 

There are a few things to keep in mind when caching a page with a SEARCH:

   * If a new topic is created that the SEARCH should find, it will not be listed.
   * If content in an existing topic is changed so it will start being found by the SEARCH, it will not be listed.
   * If the content of an already found topic changes the cache will update itself.
   
To avoid this effect you can 

   * Put the SEARCH inside a [[#dirtyarea][dirty area]]. This will prevent the SEARCH from being cached.
   * List the topic with the SEARCH in WEBDEPENDENCIES. Topics in this list will be refreshed whenever a topic in this web is editted.
   * Add a refresh button to the page to allow users to manually refresh the page.

Note that page caching doesn't only affect SEARCH. Every other information that changes without the
dependency tracker noticing it will not be rendered up-to-date on a cached page. Other candidates
are [[VarWEBLIST][WEBLIST]], as well as any plugin reading external
information, for instance results of an SQL, SOAP or LDAP query being rendered on a Foswiki topic.

This class of SEARCH-like expressions all show the same caching artefact: results are not 100% up-to-date.
At the same time these expressions are the candidates that benefit the most from caching as their computation
are the most expensive ones.
   
---++ Configuring the Cache

---+++ Choosing a database engine
The database engine used by the cache is selected using the ={Cache}{CacheManager}= *EXPERT* setting in =configure=.
   * Use =Foswiki::Cache::FileCache=  for long term caching. Cached pages will
     be stored on disk. This is the default  cache type. The required CPAN
     libraries are included with Foswiki, so it should work "out-of-the-box",
     even on hosted sites.
   * =Foswiki::Cache::BDB= uses the fast and freely available Berkeley DB.
     However it requires the Berkeley DB to be installed on the server, and
     requires the additional Perl module =BerkeleyDB= (available from CPAN).
   * Use =Foswiki::Cache::Memcached= for distributed caching on high end sites.
     Look up =memcached= in your favourite search engine for more information
     on how this powerful tool works. This option requires the
     =Cache::Memcached= Perl module, available from CPAN.
   * Use =Foswiki::Cache::MemoryLRU= for an in-memory LRU (least recently used)
     cache. This is only useful
     when used in conjunction with a persistent perl back end, such as
     =mod_perl= or =fastcgi=. Pages will be cached for the lifetime of each
     persistent backend process (but are *not* shared between different backend
     instances). 

<a name="metadatadb"></a>
---++++ The meta-data DB
The  ={Cache}{MetaCacheManager}= *EXPERT* setting in =configure= is used to
select the database implementation used to store cache meta-data (data about
the cache). This data needs to be accessed as fast as possible.

   * =Foswiki::Cache::DB_File= uses plain files to store the cache. This is the default and works "out-of-the-box".
   * =Foswiki::Cache::BDB= uses the Berkeley DB, which is recommended, but requires additional libraries to be installed (see above).

While the cached pages might be safely discarded in a page cache, meta data
about page dependencies must be stored reliably. That's why only DB_File and
Berkeley DB are selectable for the meta data cache.

---+++ Namespaces
Sometime you may want to use a database that is shared with other Foswiki
installs. In this case you need to be able to distinguish data that is stored
for this install versus other users of the database. To do this you can set the
={Cache}{Namespace}= *EXPERT* setting in =configure= to a name that is unique
for the configured system. 

---+++ Tuning
A number of options exist to help you tune the cache so that it works better
for your specific content.

---++++ Selecting topics to ignore
Some topics change so infrequently that it's worth telling the cache about them
so they can be ignored when computing dependencies. The
={Cache}{DependencyFilter}= *EXPERT* setting in =configure= defines a regular
expression that, when matched by the name of a topic, exclude the topic from
dependencies in the cache. This helps to reduce the number of topics a page
depends on, but at a higher risk that you may experience unwanted caching
effects. These effects may be seen where a cached page is delivered that does
not reflect changes made in a topic that matched the filter.

---++++ Topics that always change
Some other topics change automatically whenever _any_ topic in the same web is
saved. These are usually topics that are generated by searches over the
content. You can list the names of these topics in the
={Cache}{WebDependencies}= *EXPERT* setting in =configure=.  Whenever any topic
in the same web is saved, the cached versions of the listed topics are removed
from the cache as well.

Web dependencies can also be specified using the =WEBDEPENDENCIES= web
preference setting. If =WEBDEPENDENCIES= is set, it overrides the setting of
={Cache}{WebDependencies}= for that web. To use the preference set it to a
comma separated list of topics. No wildcards are allowed. Example  * Set
WEBPREFERENCES = !WebRss, !WebAtom, !WebTopicList, !WebIndex, !WebSearch,
!WebSearchAdvanced, !MyOwnProtectedTopic.

---++++ Compression
Modern browsers can decompress content encoded using gzip compression. You can
save a lot of network bandwidth by compressing pages, at the expense of some
server performance spent on compressing the pages. Foswiki supports content
compression when the ={HttpCompress}= option is enabled in =configure=, even
when the cache is not enabled.

Content compression makes even more sense when used with the cache, as the
cache by default stores pages on disk already compressed, so serving a
compressed page is very fast. However if you don't have =Compress::Zlib=, or
you're not using ={HttpCompress}= and do not care about disk space, you can
disable this compression by disabling the ={Cache}{Compress}= *EXPERT* option
in =configure=.

Note that only pages without any [[#dirtyarea][dirty areas]] will be
compressed. Any other page will be transmitted uncompressed.  Note also, that
compressing a page requires computation on both sides, the server performing
the compression and the browser to uncompress the received HTML before
rendering it. So only use page compression when network bandwidth is a
bottleneck.  Do not use compression on fast network connections that can easily
deliver the same content uncompressed.

---++ Technical notes
---+++ Working with browsers
The Foswiki cache works closely with the browser to optimise the use of network
bandwidth. Whenever a page is cached, Foswiki also stores its
[[http://en.wikipedia.org/wiki/HTTP_ETag][etag]], computed based on the time it
was added to the server cache. The =Etag= and =Last-Modified= headers will be
added to the response. So whenever a page is requested again, using a request
that includes the =If-None-Match= and/or =If-Modified-Since= request headers,
then Foswiki will answer with a =304 - Not modified message= and an empty body.
This tells the browser to reuse the page stored in its own client-side cache. 

---+++ Identifying what is in the cache
Depending on the values of a number of different parameters, a generated page
may have very different output.  For example, depending on the user who is
logged in, a page might be displayed very differently.

The cache has to record a this sort of environment information in order to
correctly identify pages in the cache.

   * The server serving the request (HTTP_HOST)
   * The port number of the server serving the request (HTTP_PORT)
   * The language of the current session, if any
   * All session parameters EXCEPT:
      * All those starting with an underscore
      * VALIDATION
      * REMEMBER
      * FOSWIKISTRIKEONE.*
      * VALID_ACTIONS.*
      * BREADCRUMB_TRAIL
   * All the request parameters EXCEPT:
      * All those starting with an underscore
      * refresh
      * foswiki_redirect_cache
      * logout
      * style.*
      * switch.*
      * topic

Note that this also means that users will never share cached pages among each other.
This separation is required to prevent users from seeing pages that have been rendered
with different access rights to the content than their own. For instance, a
SEARCH will show rather differnt results based on the concrete clearance on the content.
When such a page is cached it must only be retrieved for the identical user it was
generated for the first time.

----+++ Dependencies
   * {HttpCompress} and {Cache}{Compress} depend on =Compress::Zlib=
   * Cache::FileCache,>=0,cpan,Required
   * Cache::Memcached,>=0,cpan,Optional
   * Cache::MemoryCache,>=0,Optional
   * DB_File,>=0,cpan,Optional
   * !BerkeleyDB,>=0,perl,Optional, used by the BDB !CacheManager and !MetaCacheManager

<!-- %JQREQUIRE{"chili"}% -->