server.rst

kawipiko -- blazingly fast static HTTP server

kawipiko-server

>> kawipiko-server --help
>> kawipiko-server --man

--archive <path>
--archive-inmem           (memory-loaded archive file)
--archive-mmap            (memory-mapped archive file)
--archive-preload         (preload archive in OS cache)

--bind <ip>:<port>        (HTTP, only HTTP/1.1, FastHTTP)
--bind-2 <ip>:<port>      (HTTP, only HTTP/1.1, Go net/http)
--bind-tls <ip>:<port>    (HTTPS, only HTTP/1.1, FastHTTP)
--bind-tls-2 <ip>:<port>  (HTTPS, with HTTP/2, Go net/http)
--bind-quic <ip>:<port>   (HTTPS, with HTTP/3)

--http1-disable
--http2-disable
--http3-alt-svc <ip>:<port>

--tls-bundle <path>       (TLS certificate bundle)
--tls-public <path>       (TLS certificate public)
--tls-private <path>      (TLS certificate private)
--tls-self-rsa            (use self-signed RSA)
--tls-self-ed25519        (use self-signed Ed25519)

--processes <count>       (of slave processes)
--threads <count>         (of threads per process)

--index-all
--index-paths
--index-data-meta
--index-data-content

--hosts-disable           (ignore `Host` header)

--special-pages-disable
--security-headers-disable
--security-headers-tls

--seccomp-enable
--limit-descriptors <count>
--limit-memory      <MiB>
--timeout-disable

--report  --quiet  --debug

--dummy
--dummy-empty
--dummy-delay <duration>

--profile-cpu <path>
--profile-mem <path>

--version
--help          (show this short help)
--man           (show the full manual)

--sources-md5   (dump an `md5sum` of the sources)
--sources-cpio  (dump a `cpio.gz` of the sources)

--sbom  --sbom-text  --sbom-json

---

Flags

--bind <ip:port>, --bind-tls <ip:port>, --bind-2 <ip:port>, --bind-tls-2 <ip:port>, and --bind-quic <ip:port>

The IP and port to listen for requests with:

• (insecure) HTTP/1.1 for --bind, leveraging fasthttp library;
• (secure) HTTP/1.1 over TLS for --bind-tls, leveraging fasthttp library;
• (insecure) HTTP/1.1 for --bind-2, leveraging Go's net/http library; (not as performant as the fasthttp powered endpoint;)
• (secure) H2 or HTTP/1.1 over TLS for --bind-tls-2, leveraging Go's net/http; (not as performant as the fasthttp powered endpoint;)
• (secure) H3 over QUIC for --bind-quic, leveraging github.com/lucas-clemente/quic-go library; (given that H3 is still a new protocol, this must be used with caution; also one should use the --http3-alt-svc <ip:port>;)
• if one uses just --bind-tls (without --bind-tls-2, and without --http2-disabled), then the TLS endpoint is split between fasthttp for HTTP/1.1 and Go's net/http for H2;

--tls-bundle <path>, --tls-public <path>, and --tls-private <path> (optional)

If TLS is enabled, these options allows one to specify the certificate to use, either as a single file (a bundle) or separate files (the actual public certificate and the private key).

If one doesn't specify any of these options, an embedded self-signed certificate will be used. In such case, one can choose between RSA (the --tls-self-rsa flag) or Ed25519 (the --tls-self-ed25519 flag);

--http1-disable, --http2-disable

Disables that particular protocol. (It can be used only with --bind-tls-2, given that fasthttp only supports HTTP/1.)

--processes <count> and --threads <count>

The number of processes and threads per each process to start. (Given Go's concurrency model, the threads count is somewhat a soft limit, hinting to the runtime the desired parallelism level.)

It is highly recommended to use one process and as many threads as there are cores.

Depending on the use-case, one can use multiple processes each with a single thread; this would reduce goroutine contention if it causes problems. (However note that if using --archive-inmem, then each process will allocate its own copy of the database in RAM; in such cases it is highly recommended to use --archive-mmap.)

--archive <path>

The path of the CDB file that contains the archived static content. (It can be created with the kawipiko-archiver tool.)

--archive-inmem

Reads the CDB file in RAM, and thus all requests are served from RAM without touching the file-system. (The memory impact is equal to the size of the CDB archive. This can be used if enough RAM is available to avoid swapping.)

--archive-mmap

(recommended) The CDB file is memory mapped, thus reading its data uses the kernel's file-system cache, as opposed to issuing read syscalls.

--archive-preload

Before starting to serve requests, read the CDB file so that its data is buffered in the kernel's file-system cache. (This option can be used with or without --archive-mmap.)

--index-all, --index-paths, --index-data-meta, and --index-data-content

In order to serve a request kawipiko does the following:

• given the request's path, it is used to locate the corresponding resource's metadata (i.e. response headers) and data (i.e. response body) references; by using --index-paths a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource paths combined; given that the number of resources is acceptable, say up to a couple hundred thousand, one could safely use this option;)
• based on the resource's metadata reference, the actual metadata (i.e. the response headers) is located; by using --index-data-meta a RAM-based lookup table is created to eliminate a CDB read operation for this purpose; (the memory impact is proportional to the size of all resource metadata blocks combined; given that the metadata blocks are deduplicated, one could safely use this option; if one also uses --archive-mmap or --archive-inmem, then the memory impact is only proportional to the number of resource metadata blocks;)
• based on the resource's data reference, the actual data (i.e. the response body) is located; by using --index-data-content a RAM-based lookup table is created to eliminate a CDB operation operation for this purpose; (the memory impact is proportional to the size of all resource data blocks combined; one can use this option to obtain the best performance; if one also uses --archive-mmap or --archive-inmem, then the memory impact is only proportional to the number of resource data blocks;)
• --index-all enables all the options above;
• (depending on the use-case) it is recommended to use --index-paths; if --exclude-etag was used during archival, one can also use --index-data-meta;
• it is recommended to use either --archive-mmap or --archive-inmem, else (especially if data is indexed) the resulting effect is that of loading everything in RAM;

--hosts-disable

Disables the virtual-hosts feature by ignoring the Host header.

--special-pages-disable

Disables serving a few special pages internal to the server like:

/__/heartbeat
/__/kawipiko/about
/__/kawipiko/version
/__/kawipiko/manual.txt
/__/kawipiko/manual.html
/__/kawipiko/sbom.txt
/__/kawipiko/sbom.json
/__/kawipiko/sources.md5
/__/kawipiko/sources.cpio
/__/kawipiko/banners/errors/403
/__/kawipiko/banners/errors/...

--security-headers-disable

Disables adding a few security related headers:

Referrer-Policy: strict-origin-when-cross-origin
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: sameorigin

--security-headers-tls

Enables adding the following TLS related headers to the response:

Strict-Transport-Security: max-age=31536000
Content-Security-Policy: upgrade-insecure-requests

These instruct the browser to always use HTTPS for the served domain. (Useful even without HTTPS, when used behind a TLS terminator, load-balancer or proxy that do support HTTPS.)

--seccomp-enable

On Linux, and if supported, enable a strict seccomp filter to reduce the potential attack surface in case of a security issue.

The current filter is the minimal set of syscall's required to have the server working (thus quite safe). At each stage (opening the archive, indexing the archive, serving the archive) the non-required syscall's are filtered.

(At the moment the filter is quite strict and determined by experimentation. If you enable seccomp and the server is kill-ed, check auditd logs for the problematic syscall and open an issue report.)

--limit-descriptors, and --limit-memory

Constrains resource usage by configuring via setrlimit either RLIMIT_NOFILE (in case of descriptors) or both RLIMIT_DATA and RLIMIT_AS (in case of memory).

--report

Enables periodic reporting of various metrics. Also enables reporting a selection of metrics if certain thresholds are matched (which most likely is a sign of high-load).

--quiet

Disables most logging messages.

--debug

Enables all logging messages.

--dummy, --dummy-empty

It starts the server in a "dummy" mode, ignoring all archive related arguments and always responding with hello world!\n (unless --dummy-empty was used) and without additional headers except the HTTP status line and Content-Length.

This argument can be used to benchmark the raw performance of the underlying fasthttp, Go's net/http, or QUIC performance; this is the upper limit of the achievable performance given the underlying technologies. (From my own benchmarks kawipiko's adds only about ~15% overhead when actually serving the hello-world.cdb archive.)

--dummy-delay <duration>

Enables delaying each response with a certain amount (for example 1s, 1ms, etc.)

It can be used to simulate the real-world network latencies, perhaps to see how a site with many resources loads in various conditions. (For example, see an experiment I made with an image made out of 1425 tiles.)

--profile-cpu <path>, and --profile-mem <path>

Enables CPU and memory profiling using Go's profiling infrastructure.