A caching HTTP server/proxy that stores data on the local filesystem
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.



A caching HTTP server/proxy that stores data on the local filesystem


[sudo] npm install -g fs-caching-server


The fs-caching-server program installed can be used to spin up an HTTP server that acts a proxy to any other HTTP(s) server - with the added ability to cache GET and HEAD requests that match a given regex.


This will create a caching proxy that fronts Joyent's pkgsrc servers

$ mkdir cache
$ fs-caching-server -c cache/ -d -U http://pkgsrc.joyent.com
listening on
proxying requests to http://pkgsrc.joyent.com
caching matches of /\.(png|jpg|jpeg|css|html|js|tar|tgz|tar\.gz)$/
caching to /home/dave/dev/fs-caching-server/cache

-d enables debug output which can be used to determine if a file was a cache hit, cache miss, or skipped the cache completely. For example, we can request a file twice to see that it will be proxied and downloaded the first time, and the second time it will just be streamed from the local cache.

[58b93965-b7de-4669-9cb1-aff39e16a4fb] INCOMING REQUEST - GET /packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz
[58b93965-b7de-4669-9cb1-aff39e16a4fb] proxying GET to http://pkgsrc.joyent.com/packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz
[58b93965-b7de-4669-9cb1-aff39e16a4fb] saving local file to ./packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz.in-progress - - [16/May/2015:20:31:39 -0400] "GET /packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz HTTP/1.1" 200 12432 "-" "libfetch/2.0"
[58b93965-b7de-4669-9cb1-aff39e16a4fb] renamed ./packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz.in-progress to ./packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz
[ff8a1519-597f-4f9a-a999-bd05677896c2] INCOMING REQUEST - GET /packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz
[ff8a1519-597f-4f9a-a999-bd05677896c2] ./packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz is a file (cached) - streaming to client - - [16/May/2015:20:32:48 -0400] "GET /packages/SmartOS/2014Q4/x86_64/All/watch-3.2.6nb1.tgz HTTP/1.1" 200 12432 "-" "curl/7.39.0"

The lines that begin with [<uuid>] are only printed when debug (-d) is enabled - each UUID represents a unique incoming request. The first request shows the file was proxied to pkgsrc.joyent.com and streamed to both the client and the local filesystem. The second request shows the file was already present so it was streamed to the client without ever reaching out to pkgsrc.joyent.com.


$ fs-caching-server -h
usage: fs-caching-server [options]

  -c, --cache-dir <dir>     directory to use for caching data, defaults to CWD
  -d, --debug               enable debug logging to stderr
  -H, --host <host>         [env FS_CACHE_HOST] the host on which to listen, defaults to
  -h, --help                print this message and exit
  -p, --port <port>         [env FS_CACHE_PORT] the port on which to listen, defaults to 8080
  -r, --regex <regex>       [env FS_CACHE_REGEX] regex to match to cache files, defaults to \.(png|jpg|jpeg|css|html|js|tar|tgz|tar\.gz)$
  -U, --url <url>           [env FS_CACHE_URL] URL to proxy to
  -u, --updates             check npm for available updates
  -v, --version             print the version number and exit


MIT License