Reference Server

Reference server

The reference server is a configuration for Caddy, tools/server/Caddyfile, that makes Caddy a read-only server of vgl pack files, with range requests, CORS, the right caching headers and HTTPS, and that is safe to expose on a public endpoint. It needs Caddy 2.11 or later and no plugins. This page explains what it serves and refuses, how to run it locally and in production (Linux with systemd, Docker, Windows), the headers it sends, how tools/server/check.sh verifies it, and how to adapt it. For hosting in general, including object stores, see Hosting packs.

When to use it

• Development. Try packs over HTTP locally, with the samples' --pack http://..., before uploading them anywhere.
• Local networks, private and offline sites. A machine on the network serves packs to every installation, without internet access.
• Full control over a public endpoint: your own machine, your own domain, automatic certificates.

For public data read by many users, an object store or a CDN is usually simpler (Hosting packs); the reference server also works well as the origin behind a CDN.

Laying out the packs

Put the packs in one directory, each in a subdirectory of its own, exactly as the baker wrote them:

/srv/packs/
    gebco/
        pack.vglm
        base-1cd9fe31.vgls
    glo30-norway/
        pack.vglm
        base-....vgls
        shards/<y>/<x>-<digest>.vgls

A pack is then at <site>/<subdirectory>, such as https://packs.example.org/gebco, which is the URI to pass to vgl_pack_create and to the samples' --pack. The baker's .bake directory may stay or go: hidden paths are never served.

Settings

The configuration reads three environment variables:

Variable	Default	Meaning
VGL_SITE	:8080	The address to serve. A domain name (packs.example.org) serves HTTPS with a certificate Caddy obtains and renews (from Let's Encrypt or ZeroSSL), and redirects HTTP to it. A port (:8080) serves plain HTTP, for local use or behind a proxy or CDN that terminates TLS.
VGL_PACKS	packs	The directory of packs, relative to the working directory unless absolute.
VGL_CORS_ORIGIN	*	The origin that browsers may read packs from. Packs are public data, so * is usually right; name an origin (https://app.example.org) to keep other sites from using your bandwidth from browsers.

The configuration turns Caddy's admin endpoint off, so restart Caddy to change settings.

Running it locally

Install Caddy 2.11 or later (CI uses 2.11.7 from Caddy's releases). From the repository root:

# Plain HTTP on port 8080, serving the sample packs.
VGL_PACKS=samples/data/packs caddy run --config tools/server/Caddyfile

Then, from another terminal:

curl -I http://127.0.0.1:8080/gebco/pack.vglm
vgl-bake check http://127.0.0.1:8080/gebco
build/linux-gcc-shared/bin/Release/vgl-headless --pack http://127.0.0.1:8080/gebco \
    --pack http://127.0.0.1:8080/glo30-sognefjord

The samples need a vgl built with VGL_DEFAULTS_HTTP (the shared presets turn it on) to read http:// URIs (Samples). Caddy logs every request to standard output.

On Windows, in PowerShell:

$env:VGL_PACKS = "D:\packs"
caddy run --config tools\server\Caddyfile

To serve the web build of the OpenGL sample together with packs, use samples/gl/web/Caddyfile instead: it serves the page cross-origin isolated on port 8090 and the sample packs at data/packs, the same way the reference server serves them.

Running it in production

Point the domain's DNS records (A and AAAA) at the machine, open ports 80 and 443 (TCP) and 443 (UDP, for HTTP/3), and set VGL_SITE to the domain:

VGL_SITE=packs.example.org VGL_PACKS=/srv/packs caddy run --config tools/server/Caddyfile

Port 80 is needed to obtain certificates and to redirect to HTTPS. Behind a CDN or proxy that terminates TLS, set VGL_SITE to a port instead and let the CDN connect to it.

Linux with systemd

Install the Caddyfile as /etc/caddy/Caddyfile and run Caddy as an unprivileged user with read-only access to the system:

[Unit]
Description=vgl pack server
After=network-online.target
Wants=network-online.target

[Service]
User=caddy
Group=caddy
Environment=VGL_SITE=packs.example.org VGL_PACKS=/srv/packs
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/caddy
Restart=on-failure

[Install]
WantedBy=multi-user.target

Save it as, for example, /etc/systemd/system/vgl-packs.service, then:

sudo systemctl daemon-reload
sudo systemctl enable --now vgl-packs
journalctl -u vgl-packs -f      # the access log

ReadWritePaths=/var/lib/caddy is where Caddy keeps its certificates (the caddy user's data directory as Linux packages of Caddy set it up). If your distribution's package already installs a caddy.service, disable it or replace its Caddyfile, so the two do not compete for ports 80 and 443.

Docker

The repository has no Dockerfile; Caddy's official image runs the configuration as it is. The caddy-data volume keeps the certificates across restarts:

docker run -d --name vgl-packs --restart unless-stopped \
    -p 80:80 -p 443:443 -p 443:443/udp \
    -e VGL_SITE=packs.example.org -e VGL_PACKS=/srv/packs \
    -v /srv/packs:/srv/packs:ro \
    -v "$PWD/tools/server/Caddyfile:/etc/caddy/Caddyfile:ro" \
    -v caddy-data:/data \
    caddy:2.11.7

The packs are mounted read-only. For local use, publish one port and leave VGL_SITE at its default: -p 8080:8080 without -e VGL_SITE=....

Windows

Caddy runs on Windows from the same configuration. Set the variables and run it as above, or run it as a Windows service with a service wrapper, as Caddy's documentation describes; set VGL_SITE, VGL_PACKS (an absolute path) and, if needed, VGL_CORS_ORIGIN in the service's environment. Allow ports 80 and 443 through Windows Defender Firewall for a public endpoint, or only the chosen port for a local network. The repository's checks of the reference server run on Linux; Windows is not covered by CI.

What it serves and refuses

Request	Answer
GET or HEAD of <pack>/pack.vglm or *.vgls	The file, as application/octet-stream, with ranges
Range: bytes=a-b or bytes=a-	206 Partial Content
A range past the end of the file	416
Several ranges in one request (bytes=0-1,4-5)	416: vgl never asks for them, and they let a client make the server do much work for one small request
If-None-Match with the current ETag	304 Not Modified
OPTIONS (a CORS preflight)	204 with the CORS headers
Any other method (POST, PUT, DELETE, ...)	405, with Allow: GET, HEAD, OPTIONS
Directories, directory listings, other files, hidden files (any path with /.), paths outside the pack directory (.., encoded or not), missing files	404

It never compresses responses, and it does not name itself in a Server header.

Headers

On every response:

Header	Value	Why
Access-Control-Allow-Origin	VGL_CORS_ORIGIN (*)	Browsers on other sites may read packs
Access-Control-Expose-Headers	Accept-Ranges, Content-Length, Content-Range, ETag	Scripts may see the range headers
Cross-Origin-Resource-Policy	cross-origin	Cross-origin isolated pages, as vgl's web build is, may load packs however they ask
X-Content-Type-Options	nosniff	Browsers do not guess content types
Referrer-Policy	no-referrer
Strict-Transport-Security	max-age=31536000	Over HTTPS only

On pack files:

File	Cache-Control	Why
*/pack.vglm	public, max-age=60	A manifest changes in place when a pack is rebaked, so caches check it again after a minute; the ETag makes the recheck a 304
*.vgls	public, max-age=31536000, immutable	A shard's name holds a digest of its content, so caches may keep it forever

On preflight requests (OPTIONS): Access-Control-Allow-Methods: GET, HEAD, OPTIONS, Access-Control-Allow-Headers: Range, If-None-Match, If-Range and Access-Control-Max-Age: 86400.

The server also limits slow clients: 10 seconds to read request headers and bodies, 2 minutes to write a response, 2 minutes for idle connections, and 16 KB of request headers.

How check.sh verifies it

tools/server/check.sh checks the configuration end to end. It needs bash, curl, a Caddy binary and the headless sample built with VGL_DEFAULTS_HTTP:

tools/server/check.sh "$(command -v caddy)" build/linux-gcc-shared/bin/Release/vgl-headless [port]

The port defaults to 18080. The script:

1. Copies the sample packs from samples/data/packs to a temporary directory, and plants files the server must not hand out next to them: a hidden .hidden.vgls, a notes.txt inside a pack, and an outside.vgls outside the pack directory.
2. Starts Caddy with the Caddyfile on that directory.
3. Checks what a pack reader needs: 200 for the manifest (GET and HEAD); 206 for bytes=0-15 and bytes=16-; 416 past the end; the exact Content-Range and Content-Length; Accept-Ranges: bytes; the content type, nosniff, both Cache-Control values, Access-Control-Allow-Origin: * and Cross-Origin-Resource-Policy; an ETag and a 304 for it; a 204 preflight with the allowed headers; and no Server header.
4. Checks what it refuses: 405 for POST, PUT and DELETE; 416 for several ranges; 404 for the root, a pack directory with and without a slash, the planted files, and path traversal attempts (../, %2e%2e, ..%2f), and for a missing shard.
5. Loads the sample packs (GEBCO, GLO-30 Sognefjord, EGM2008, Blue Marble) with the headless sample, once from disk and once over HTTP from the server, and requires the heights it prints at its places to be identical.

It prints The reference server passed its checks. and exits with 0, or lists each failure with the end of Caddy's log and exits with 1. CI runs it on Linux with Caddy 2.11.7, downloaded at a pinned checksum, after the linux-gcc-shared build (.github/workflows/ci.yml).

For your own deployment, vgl-bake check <url> performs the client-side checks against any host, including one running this configuration (Hosting packs).

Operating it on a public endpoint

The configuration covers what a web server can enforce on its own. Also:

• Limit request rates. Caddy has no rate limiting without plugins. Put a CDN in front (it caches shards at its edge and absorbs floods), or limit connections per address in the firewall.
• Keep Caddy updated, and the operating system with it. Watch Caddy's releases for security fixes.
• Give Caddy read-only access to the packs, and run it as an unprivileged user. Caddy follows symbolic links, so keep links that point outside the pack directory out of it.
• Open only ports 80 and 443 (and UDP 443 for HTTP/3).
• Mind the logs. Access logs hold IP addresses, which are personal data in the EU. Keep them only as long as needed, or remove the log block.
• Publish the attribution that each dataset's license requires; the runtime reads it from the manifests (vgl_context_attribution_get), so applications can show it.

Adapting it

The Caddyfile is short and commented; copy it and change what you need. Common changes:

• Restrict browser origins. Set VGL_CORS_ORIGIN=https://app.example.org. The variable takes one origin, sent as it is; for several origins, echo the request's Origin when it is on an allow list (with a matcher on the Origin header and Vary: Origin), or set the rule at a CDN.
• Different cache lifetimes. Change the Cache-Control of the @manifest matcher (keep it short) or the @shard matcher (keep it long and immutable; shards never change).
• Serve packs under a path prefix, next to other content on the same site. Wrap the pack routes in handle_path /packs/* { ... }, as samples/gl/web/Caddyfile does for /data/packs/*, and keep the respond 404 fallback inside it.
• Serve a web application from the same site. Follow the web sample's Caddyfile: it adds Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp for the page, which the web build needs for threads (Web builds).
• No logs, or logs to a file. Remove the log block, or change its output to file <path>.
• Behind a CDN or reverse proxy. Set VGL_SITE to a port such as :8080. Caddy then serves plain HTTP and does not send Strict-Transport-Security, so set it at the CDN.
• Rate limiting. Add a rate-limiting plugin to a custom Caddy build, or do it in the firewall or CDN.

Whatever you change, keep these properties, which vgl relies on: single byte ranges with 206, no compression of pack files, application/octet-stream, CORS for Range if browsers read the packs, and long caching only for shards. Then run tools/server/check.sh (for the configuration) and vgl-bake check (for the deployment) again.

See also Hosting packs, Pack format overview and Baker.