ruxen v0.1.1

ruxen 0.1.1: security fixes and a large batch of nginx-compatibility work since 0.1.0. ruxen is an experimental nginx-compatible HTTP server and reverse proxy written from scratch in Rust: Linux-only, HTTP/1.1, thread-per-core on io_uring (monoio), TLS via rustls.

Warning

This is an experimental release. It is not production-ready and is not a drop-in replacement for nginx.

Security

This release fixes six problems that were reported privately. 0.1.0 is affected by all of them; upgrade if you run it. Details are in the linked advisories.

• High: GHSA-5gfx-2fj2-cwpc. After an upstream answered 101 Switching Protocols, its connection went back to the upstream keep-alive pool. A later request from another client could be written into the first client's switched connection. Affects proxy_pass to an upstream with keepalive whose backend upgrades connections (WebSocket recipes).
• High: GHSA-xq4w-j54g-x6ww. With proxy_intercept_errors on, an error_page whose target is an absolute URL, or a variable that renders empty, made a worker panic on any request that got that status from the upstream. With one worker the server exited. Such targets now redirect, or leave the upstream response as it is, as in nginx.
• Medium: GHSA-qgh6-g2wj-7c9p. Request bodies spilled to disk were world-readable files with predictable names in the system temp directory. They are now created 0600 and exclusively, in client_body_temp_path (now honoured at http level) or in a private 0700 directory.
• Medium: GHSA-qxw8-jfcm-4qrj. disable_symlinks on|if_not_owner was accepted and ignored, so symlinks under the root were served. ruxen can't enforce it yet, so it is now refused at startup ([emerg]) instead.
• Medium: GHSA-9q8r-38xr-vm9x. With ssl_session_cache on, a table of session timestamps grew with every full handshake and was never pruned.
• Low: GHSA-9rpw-5wjv-g3v6. Upstream response bodies were buffered whole with no size limit, so one large response could take a worker's memory. They are now capped at 64 MiB, whatever their framing, and a larger one gets a 502.

Behaviour change: configurations with disable_symlinks on or if_not_owner no longer start. Remove the directive or set it to off until ruxen implements it.

Install

Prebuilt binary (x86_64 Linux, glibc 2.35 or newer):

curl -LO https://github.com/gvozdetsky/ruxen/releases/download/v0.1.1/ruxen-v0.1.1-x86_64-linux-gnu.tar.gz
sha256sum -c --ignore-missing SHA256SUMS   # after downloading SHA256SUMS from this release
tar xzf ruxen-v0.1.1-x86_64-linux-gnu.tar.gz
cd ruxen-v0.1.1-x86_64-linux-gnu
./ruxen -t -c "$PWD/examples/minimal.conf"

ruxen-x86_64-linux-gnu.tar.gz is the same tarball under a version-less name; releases/latest/download/ruxen-x86_64-linux-gnu.tar.gz always gets the newest release.

From crates.io (needs cmake and a C compiler for rustls's aws-lc-rs backend):

cargo install ruxen --locked

Requirements: a Linux kernel with io_uring enabled. In Docker, run with --security-opt seccomp=unconfined: the default seccomp profile blocks io_uring.

What's new since 0.1.0

nginx compatibility

• Connections and requests:
  • listen … proxy_protocol (PROXY protocol v1 and v2, $proxy_protocol_*);
  • listen port ranges;
  • worker_connections;
  • client_header_timeout, client_body_timeout, send_timeout;
  • lingering close after a refused upload;
  • client_max_body_size above 1 MiB, with bodies spilled to disk and an over-limit Content-Length refused before the body is read.
• Request processing:
  • the server-level rewrite phase (rewrite, set, if, return, break in server {});
  • internal locations;
  • recursive_error_pages;
  • error_page for requests refused before a location;
  • limit_rate, limit_rate_after, $limit_rate;
  • map with hostnames, escaped keys and case-insensitive exact keys;
  • unknown variables rejected at startup like nginx.
• Reverse proxy:
  • failover as nginx does it (proxy_next_upstream accounting for 403/404 and the last try, proxy_intercept_errors on the last try, retries of non-idempotent requests);
  • proxy_redirect with the implicit default;
  • proxy_hide_header / proxy_pass_header;
  • proxy_set_body;
  • X-Accel-Redirect, X-Accel-Limit-Rate and proxy_ignore_headers;
  • upstream framing checked like nginx;
  • $upstream_addr, $upstream_status and the other per-attempt variables;
  • $proxy_port;
  • $proxy_host without a default :80;
  • upstreams of more than 64 servers;
  • least_conn is now claimed in -V.
• TLS: ssl_session_cache (shared by the workers), ssl_session_tickets and ssl_session_timeout with nginx's defaults; X.509 v1 certificates.
• Logging:
  • access log: nginx's combined format, $request, - and escaping; lines for rejected requests; the final $uri; set variables;
  • error log: honoured at http and top level; failed file lookups and upstream errors worded like nginx, each logged with its own request; PROXY protocol failures;
  • syslog: targets for access_log and error_log;
  • log reopening on SIGUSR1.
• Command line:
  • -s stop|quit|reopen;
  • fast shutdown on SIGTERM / SIGINT;
  • -t prints nginx's success lines;
  • startup errors are [emerg] lines, never a panic;
  • a missing root directory is a 404 per request, as in nginx.

The full list is the merged pull requests between v0.1.0 and v0.1.1.

Compatibility

ruxen is checked against the upstream nginx-tests suite: 63 of the 105 test files that ruxen opts into pass end-to-end (0.1.0: 49 of 104). The other 400 files are skipped because ruxen -V only claims the modules ruxen implements. Per-file status: NGINX_TEST_PROGRESS.md.

Performance

ruxen / nginx 1.24 throughput from interleaved A/B pairs (bench/scripts/pair.sh): 6 pairs per scenario, geometric mean of the per-pair ratios, wrk -t16 -c512 unless the scenario says otherwise, no errors or timeouts in any run. One laptop (i9-13900HX, Linux 7.0), client and server on the same machine over loopback: indicative, not a general claim. Method and scenarios: bench/README.md.

scenario	ruxen / nginx	0.1.0
return 200, keep-alive (m1_hello)	98.8%	97.6%
return 200, no keep-alive (m1_no_keepalive)	95.4%	94.7%
static 1 KiB, keep-alive (m3_static_1k)	101.6%	102.4%
static 1 KiB, no keep-alive (m3_static_1k_no_keepalive)	93.0%	90.7%
static 8 KiB (static_8k)	98.9%	100.8%
304, If-None-Match (m5_conditional_304)	94.9%	94.7%
206 range (m5_range_206)	102.3%	105.9%
streamed 1 MiB (m19_stream_1m)	101.1%	101.4%
streamed 128 MiB (m19_stream_128m)	96.8%	97.6%
six literal add_header (add_header_many)	100.2%	100.6%
auth_basic (auth_basic_hello)	112.5%	105.3%
rewrite … last (rewrite_hello)	96.7%	95.7%
map (vars_map)	95.0%	96.1%
proxy_pass with upstream keep-alive (proxy_hello)	95.2%	96.6%
TLS 1.3 return 200 (tls_hello)	109.5%	110.7%

All 22 scenarios: 18 at or above 95% of nginx. Below it are m1_keepalive_requests_1 (94.3%), m3_static_1k_no_keepalive (93.0%), m5_conditional_304 (94.9%) and m19_stream_1m_low_concurrency (93.1%, with a 88–100% spread between pairs). The 0.1.0 column was measured on another day. Comparing the 0.1.0 and 0.1.1 binaries directly (12 interleaved pairs) gives 99–101% on vars_map, static_8k, m5_range_206, m19_stream_1m_low_concurrency and m1_keepalive_requests_1, and 99.0% on proxy_hello: the proxy path is about 1% slower than in 0.1.0, the cost of the proxy features added in this release (#177).

Known limitations

• include does not expand globs (include conf.d/*.conf;).
• With sendfile on, the file side of the transfer is synchronous.
• sendfile_max_chunk is not configurable.
• Proxied responses are buffered whole before they are sent (up to 64 MiB); streaming them is planned.
• No configuration reload: -s reload is refused, restart instead.
• -V reports nginx version: nginx/1.29.2 (followed by ruxen version: ruxen/0.1.1): the nginx-tests harness reads that line and the configure arguments to decide which tests apply.

How to help

• Found a config where nginx and ruxen behave differently? Open an issue with the config, the request and both responses.
• Security problems: please report them privately, see SECURITY.md.
• Setup, tests and benchmarks: CONTRIBUTING.md.

ruxen v0.1.0

First public release of ruxen, an experimental nginx-compatible HTTP server and reverse proxy written from scratch in Rust: Linux-only, HTTP/1.1, thread-per-core on io_uring (monoio), TLS via rustls.

Warning

This is an experimental release. It is not production-ready and is not a drop-in replacement for nginx.

Install

Prebuilt binary (x86_64 Linux, glibc 2.35 or newer):

curl -LO https://github.com/gvozdetsky/ruxen/releases/download/v0.1.0/ruxen-v0.1.0-x86_64-linux-gnu.tar.gz
sha256sum -c SHA256SUMS   # after downloading SHA256SUMS from this release
tar xzf ruxen-v0.1.0-x86_64-linux-gnu.tar.gz
cd ruxen-v0.1.0-x86_64-linux-gnu
./ruxen -t -c "$PWD/examples/minimal.conf"
./ruxen -c "$PWD/examples/minimal.conf"
curl -i http://127.0.0.1:8080/

From crates.io (needs cmake and a C compiler for rustls's aws-lc-rs backend):

cargo install ruxen --locked

Requirements: a Linux kernel with io_uring enabled (developed and tested on 6.x–7.0). In Docker, run with --security-opt seccomp=unconfined: the default seccomp profile blocks io_uring.

What works

nginx-style configuration with http / server / location blocks, static files (including sendfile, ranges, conditional requests, autoindex), return / rewrite / if / set / map / split_clients, error_page, try_files, add_header, auth_basic, access and error logs, reverse proxying with upstream groups and keep-alive pools, and TLS 1.2/1.3 termination with SNI. The full list is in the README.

Not included on purpose

HTTP/2 and HTTP/3, the module system, HTTP caching, TLS to upstreams (proxy_ssl_*), configuration reload without restart.

Compatibility

ruxen is checked against the upstream nginx-tests suite: 49 of the 104 test files that ruxen opts into pass end-to-end. The other 401 files are skipped because ruxen -V only claims the modules ruxen implements, so the harness doesn't run tests for stream, mail, HTTP/2, cache and so on. Per-file status: NGINX_TEST_PROGRESS.md.

Performance

ruxen / nginx 1.24 throughput from interleaved A/B pairs: 6 pairs per scenario (12 for tls_hello, re-measured on the release build after the TLS idle-timeout change), geometric mean of the per-pair ratios, wrk -t16 -c512 unless the scenario says otherwise, no errors or timeouts in any run. One laptop (i9-13900HX, Linux 7.0), client and server on the same machine over loopback — indicative, not a general claim. Method and scenarios: bench/README.md.

scenario	ruxen / nginx
return 200, keep-alive (m1_hello)	97.6%
return 200, no keep-alive (m1_no_keepalive)	94.7%
static 1 KiB, keep-alive (m3_static_1k)	102.4%
static 1 KiB, no keep-alive (m3_static_1k_no_keepalive)	90.7%
static 8 KiB (static_8k)	100.8%
304, If-None-Match (m5_conditional_304)	94.7%
206 range (m5_range_206)	105.9%
streamed 1 MiB (m19_stream_1m)	101.4%
streamed 128 MiB (m19_stream_128m)	97.6%
six literal add_header (add_header_many)	100.6%
auth_basic (auth_basic_hello)	105.3%
rewrite … last (rewrite_hello)	95.7%
map (vars_map)	96.1%
proxy_pass with upstream keep-alive (proxy_hello)	96.6%
TLS 1.3 return 200 (tls_hello)	110.7%

All 22 scenarios: 19 at or above 95% of nginx. The weakest is short-lived connections: without keep-alive, the per-connection cost (accept, setup, teardown on io_uring) is higher than nginx's on epoll.

Known limitations

• On proxied responses the upstream's Server header is passed through; nginx replaces it with its own.
• include does not expand globs (include conf.d/*.conf;).
• With sendfile on, the file side of the transfer is synchronous.
• sendfile_max_chunk is not configurable.
• -V reports nginx version: nginx/1.29.2 (followed by ruxen version: ruxen/0.1.0): the nginx-tests harness reads that line and the configure arguments to decide which tests apply.

How to help

• Found a config where nginx and ruxen behave differently? Open an issue with the config, the request and both responses.
• Good starting points: #14 (benchmarks on other hardware), #15, #16, #17, #18.
• Setup, tests and benchmarks: CONTRIBUTING.md.