XSendFileContrib.txt

%META:TOPICINFO{author="ProjectContributor" comment="reprev" date="1357654251" format="1.1" reprev="3" version="3"}%
---+!! %TOPIC%
%TOC%

---++ Introduction
This package implements a more efficient way to send static files using a web application
like Foswiki. A standard configuration of a Foswiki uses the =viewfile= service
to send static files over to the browser thus enforcing access rights on them. In doing
so it reads the static file into memory completely before handing it over to the HTTP web server
which takes over responsibility to contact the browser. This of course is far from optimal
as it introduces a lot of overhead by copying the static file several times in memory. The impact
on the overall system performance is even higher for large files being downloaded from a Foswiki
server. For now the only alternative solution to this problem is not to protect static files
by Foswiki's access control at all, and let the HTTP web server do its job on static files all
on its own. Web servers are in fact quite good in sending static files, even large ones. So the
best you can do is to leave this job to them.

Still you might require access control, even on large static files. 

There's a quite unknown yet very important feature in Apache2, Nginx and Lighttpd, called "xsendfile", to interact
with an upstream web application before sending a static file. This feature can be used to solve
the above problem: it lets you control access to static files while still using the HTTP web
server to do the heavy lifting.

This basically works like this:

   * client requests an url 
   * the web server calls an upstream web application for this url
   * the web app performs its specific actions and returns a specific HTTP header in its response, e.g. =X-Lighttp-send-file=, but _not_ the data being requested, when access is granted, or an HTTP error otherwise
   * the web server parses the HTTP headers returned by the web app looking for the X-Send-File header
   * if it finds the X-Send-File header then the will the web serve transfer the static file this header points to over to the client
   * otherwise it will return the response generated by the web app as usual

---++ Configuring Foswiki 
<nop>%TOPIC% comes with a separate service script =.../bin/xsendfile= replacing the standard =.../bin/viewfile=. It differs from viewfile only in two respects

   * any HTTP error is delivered immediately, that is without requiring Foswiki to render an error page while processing an exception. 
   * instead of copying over the static file to the downstream HTTP web server, it generates an empty response with the approrpiate X-Send-File header set

There are two parameters that need to be adjusted according to your web server's type and configuration:

   * =$Foswiki::cfg{XSendFileContrib}{Header}= ... the name of the HTTP header to trigger the X-Send-File feature in your web server (see below)
   * =$Foswiki::cfg{XSendFileContrib}{Location}= ... the uri prefix that the web server serves the actual static file from 

---++ Configuring Lighttpd
(see http://redmine.lighttpd.net/projects/1/wiki/X-LIGHTTPD-send-file for lighttpd)

The X-Send-File header for Lighttpd is called *X-LIGHTTPD-send-file* and points to the file path on disk that the web server should send to the client.

If you are already using Foswiki behind a lighttpd web server, all you need to do is to add ="allow-x-send-file" => "enable"= to your fastcgi stanza

<verbatim>
url.rewrite-once += ( "^/pub/((?!System|Applications|images|cache).*)/(.*?)$" => "/bin/xsendfile/$1/$2" )

$HTTP["url"] =~ "^/bin/" {
  alias.url += ( "/bin" => "/path/to/bin/foswiki.fcgi" )

  fastcgi.server = ( ".fcgi" => ((
...
      "allow-x-send-file" => "enable"
    ),
  ))
}
$HTTP["url"] =~ "^/bin/xsendfile)" {
  expire.url = ( "" => "access 12 hours")
}
</verbatim>

---++ Configuring Nginx
(see http://wiki.nginx.org/XSendfile, http://wiki.nginx.org/X-accel)

In Nginx this feature is called X-Accel-Redirect. Yet the idea is the same. Requirements on the upstream web applications are almost identical
besides using a different X-Send-File header called *X-Accel-Redirect*.

In addition Nginx requires a special "location" stanza for internal use only. Protected files will be served from there one the web app
checked the original uri the.

<verbatim>
location ~ ^/pub/(System|Applications|images|cache)/ {
  root /path/to/foswiki;
  expires 12h;
  gzip_static on;
}

location /pub {
  rewrite ^/pub/(.*)$ /bin/xsendfile/$1;
}

location /protected_files {
   internal;
   alias /path/to/foswiki/pub/;
}
</verbatim>

---++ Configuring Apache2
Apache itself requires an additional simple module that processes an *X-Sendfile* header.

See https://tn123.org/mod_xsendfile/ for more information on how to compile =mod_xsendfile= and how to configure apache.
---++ Installation
%$INSTALL_INSTRUCTIONS%

See above for more information on how to configure Foswiki and your web server.

---++ Info
<!--
One line description, required for extensions repository catalog.
   * Set SHORTDESCRIPTION = %$SHORTDESCRIPTION%
-->

|  Author: | Michael Daum |
|  Copyright &copy;: | 2013, Michael Daum http://michaeldaumconsulting.com |
|  License: | GPL ([[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]]) |
|  Dependencies: | %$DEPENDENCIES% |
|  Version: | %$VERSION% |
|  Change History: | <!-- versions below in reverse order -->&nbsp; |
|  01 Nov 2013 | added support for if-modified-since http headers |
|  22 May 2013 | using mime-magic as a fallback in case file extensions don't unveil the mime-type |
|  28 Mar 2013 | implemented {<nop>AccessRules} to allow any kind of access control list for attachments |
|  1.00: | Initial version |
|  Home: | http://foswiki.org/Extensions/%TOPIC% |
|  Support: | http://foswiki.org/Support/%TOPIC% |