Support for serving pre-compressed files

[joseluisq]

Description

This PR provides support for serving pre-compressed files via a new boolean compression-static option.
It checks for existing pre-compressed (.gz or .br) files on disk and the preferred-encoding is determined via the accept-encoding header value.

When the compression-static option is enabled and the pre-compressed file is found then it is served directly.
Otherwise, if the pre-compressed file is not found then SWS just continues the normal workflow (trying to serve the original file requested).
For example, if the compression option was also enabled then the requested file be compressed on the fly.

Motivation and Context

Feature request #114

How Has This Been Tested?

static-web-server -p=8787 -d=/var/www -g=trace --compression-static=true`

Relevant log entries

2022-09-22T21:30:12.904102Z  INFO static_web_server::handler: incoming request: method=GET uri=/downloads/Capture5.png
2022-09-22T21:30:12.904218Z TRACE static_web_server::static_files: dir: base="/var/www", route="downloads/Capture5.png"
2022-09-22T21:30:12.904295Z TRACE static_web_server::compression_static: preparing pre-compressed file path variant of /var/www/downloads/Capture5.png
2022-09-22T21:30:12.904509Z TRACE static_web_server::compression_static: getting metadata for pre-compressed file variant /var/www/downloads/Capture5.png.gz
2022-09-22T21:30:12.904746Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-09-22T21:30:12.904932Z TRACE static_web_server::static_files: file found: "/var/www/downloads/Capture5.png.gz"
2022-09-22T21:30:12.904983Z TRACE static_web_server::compression_static: pre-compressed file variant found, serving it directly
2022-09-22T21:30:12.905095Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
2022-09-22T21:30:12.905836Z TRACE encode_headers: hyper::proto::h1::role: Server::encode status=200, body=Some(Unknown), req_method=Some(GET)
2022-09-22T21:30:12.905965Z TRACE encode_headers: hyper::proto::h1::role: close time.busy=138µs time.idle=35.4µs
2022-09-22T21:30:12.906236Z DEBUG hyper::proto::h1::io: flushed 242 bytes

Full log entries

  # 022-09-22T21:30:02.882007Z  INFO static_web_server::logger: logging level: trace
  # 2022-09-22T21:30:02.882113Z DEBUG static_web_server::server: initializing tokio runtime with multi thread scheduler
  # 2022-09-22T21:30:02.882528Z TRACE mio::poll: registering event source with poller: token=Token(0), interests=READABLE    
  # 2022-09-22T21:30:02.883490Z  INFO static_web_server::server: server bound to tcp socket [::]:8787
  # 2022-09-22T21:30:02.883590Z  INFO static_web_server::server: runtime worker threads: 4
  # 2022-09-22T21:30:02.883626Z  INFO static_web_server::server: security headers: enabled=false
  # 2022-09-22T21:30:02.883648Z  INFO static_web_server::server: auto compression: enabled=true
  # 2022-09-22T21:30:02.883667Z  INFO static_web_server::server: compression static: enabled=true
  # 2022-09-22T21:30:02.883687Z  INFO static_web_server::server: directory listing: enabled=false
  # 2022-09-22T21:30:02.883708Z  INFO static_web_server::server: directory listing order code: 6
  # 2022-09-22T21:30:02.883726Z  INFO static_web_server::server: cache control headers: enabled=true
  # 2022-09-22T21:30:02.883779Z  INFO static_web_server::server: basic authentication: enabled=false
  # 2022-09-22T21:30:02.883799Z  INFO static_web_server::server: log remote address: enabled=false
  # 2022-09-22T21:30:02.883817Z  INFO static_web_server::server: redirect trailing slash: enabled=true
  # 2022-09-22T21:30:02.883835Z  INFO static_web_server::server: grace period before graceful shutdown: 0s
  # 2022-09-22T21:30:02.883910Z TRACE mio::poll: registering event source with poller: token=Token(1), interests=READABLE | WRITABLE    
  # 2022-09-22T21:30:02.883961Z TRACE mio::poll: registering event source with poller: token=Token(2), interests=READABLE | WRITABLE    
  # 2022-09-22T21:30:02.884404Z TRACE mio::poll: registering event source with poller: token=Token(3), interests=READABLE | WRITABLE    
  # 2022-09-22T21:30:02.884651Z  INFO Server::start_server{addr_str="[::]:8787" threads=4}: static_web_server::server: close time.busy=0.00ns time.idle=36.0µs
  # 2022-09-22T21:30:02.884728Z  INFO static_web_server::server: listening on http://[::]:8787
  # 2022-09-22T21:30:02.884758Z  INFO static_web_server::server: press ctrl+c to shut down the server
  # 2022-09-22T21:30:12.901112Z TRACE mio::poll: registering event source with poller: token=Token(4), interests=READABLE | WRITABLE    
  # 2022-09-22T21:30:12.901693Z TRACE hyper::proto::h1::conn: Conn::read_head
  # 2022-09-22T21:30:12.901826Z TRACE hyper::proto::h1::io: received 444 bytes
  # 2022-09-22T21:30:12.902149Z TRACE parse_headers: hyper::proto::h1::role: Request.parse bytes=444
  # 2022-09-22T21:30:12.902341Z TRACE parse_headers: hyper::proto::h1::role: Request.parse Complete(444)
  # 2022-09-22T21:30:12.903092Z TRACE parse_headers: hyper::proto::h1::role: close time.busy=975µs time.idle=46.8µs
  # 2022-09-22T21:30:12.903534Z DEBUG hyper::proto::h1::io: parsed 10 headers
  # 2022-09-22T21:30:12.903849Z DEBUG hyper::proto::h1::conn: incoming body is empty
  2022-09-22T21:30:12.904102Z  INFO static_web_server::handler: incoming request: method=GET uri=/downloads/Capture5.png
  2022-09-22T21:30:12.904218Z TRACE static_web_server::static_files: dir: base="/var/www", route="downloads/Capture5.png"
  2022-09-22T21:30:12.904295Z TRACE static_web_server::compression_static: preparing pre-compressed file path variant of /var/www/downloads/Capture5.png
  2022-09-22T21:30:12.904509Z TRACE static_web_server::compression_static: getting metadata for pre-compressed file variant /var/www/downloads/Capture5.png.gz
  2022-09-22T21:30:12.904746Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
  2022-09-22T21:30:12.904932Z TRACE static_web_server::static_files: file found: "/var/www/downloads/Capture5.png.gz"
  2022-09-22T21:30:12.904983Z TRACE static_web_server::compression_static: pre-compressed file variant found, serving it directly
  2022-09-22T21:30:12.905095Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Init, keep_alive: Busy }
  2022-09-22T21:30:12.905836Z TRACE encode_headers: hyper::proto::h1::role: Server::encode status=200, body=Some(Unknown), req_method=Some(GET)
  2022-09-22T21:30:12.905965Z TRACE encode_headers: hyper::proto::h1::role: close time.busy=138µs time.idle=35.4µs
  2022-09-22T21:30:12.906236Z DEBUG hyper::proto::h1::io: flushed 242 bytes
  # 2022-09-22T21:30:12.906279Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.906427Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.906493Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.906655Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.906696Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.906807Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.906855Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.906992Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.907035Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.907135Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.907177Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.907297Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.907338Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.907468Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.907519Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.907699Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.907761Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.907911Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.907969Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.908186Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.908306Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.908773Z TRACE hyper::proto::h1::encode: encoding chunked 8192B
  # 2022-09-22T21:30:12.908936Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=8200
  # 2022-09-22T21:30:12.909120Z DEBUG hyper::proto::h1::io: flushed 8200 bytes
  # 2022-09-22T21:30:12.909165Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: KeepAlive, writing: Body(Encoder { kind: Chunked, is_last: false }), keep_alive: Busy }
  # 2022-09-22T21:30:12.909309Z TRACE hyper::proto::h1::encode: encoding chunked 5565B
  # 2022-09-22T21:30:12.909367Z TRACE hyper::proto::h1::io: buffer.queue self.len=0 buf.len=5573
  # 2022-09-22T21:30:12.909460Z TRACE hyper::proto::h1::io: buffer.queue self.len=5573 buf.len=5
  # 2022-09-22T21:30:12.909590Z DEBUG hyper::proto::h1::io: flushed 5578 bytes
  # 2022-09-22T21:30:12.909629Z TRACE hyper::proto::h1::conn: flushed({role=server}): State { reading: Init, writing: Init, keep_alive: Idle }

Screenshots (if appropriate):

No

[joseluisq]

Released on v2.12.0