Split one big HTTP/Range request to multiple subrange requesets
C Python Shell
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
test Add missed files Jul 28, 2014
.gitignore Initial commit May 6, 2014
.travis.yml Fix travis test_subrange Jul 25, 2014
LICENSE Initial commit May 6, 2014
README.md Fix README Aug 7, 2014
config Change project name to ngx_http_subrange_module Apr 1, 2014
ngx_http_subrange_module.c
ngx_http_subrange_module.h Update version to 0.6.1 Jul 17, 2014
release.sh

README.md

ngx_http_subrange_module

Build Status

Split one big download file request to multiple subrange requests to avoid geting too much data from upstream at one time. Install:

Configure nginx with --add-module=/path/to/ngx_http_subrange_module

Directive:

syntax : subrange size
default: subrange 0, 0 means disable
context: http, server, location

Example:

  location /{  
    root html;  
    subrange 10k;  
  }

Introduction:

When nginx is used as a reverse proxy for file downloading service, it will always run out of the bandwidth between nginx and upstream when the user requests a very large file. This is because nginx fetch a whole file at a time and buffer the left data that the client can not read in time. The bandwidth would be used up and the disk iowait would be high.

Nginx has an option to turn off the buffer mechanism, for example set proxy_buffering off or fastcgi_buffering off, which directive depends on the type of your upstream. However, this brings it with a problem. If your upstream is PHP or Java, it will block your PHP fastcgi process or yourJava threads, especially when the client is downloading a very large file and even worse he has a terrible speed.

The subrange module is created to solve this problem. It splits your HTTP requests. When you want to download an 1G file, the module will try to download a chunk of the file from the upstream, for example downloads 5M first, and then the next 5M, until the client receives the whole file. The whole process is non-sensible to client. You can set the chunk size in the nginx configuration file.

The module sets the HTTP Range header to perform a Range request to get a chunk from the upstream. So the supporting of Range request is needed by upstream. Supporting Range is easy, all standard HTTP servers like nginx/apache have implemented it. It is trivial to implement it yourself.

We just have one directive subrange which sets the size of chunk being fetched at a time. The directive takes a size as the parameter(1024 or 1k), or a variable as well. 0 meas disable subrange.

set $size 10m;
location /download{
    subrange $size;
}