Skip to content

v1.5.0 - post-quantum readiness

Choose a tag to compare

@marcoome marcoome released this 21 Sep 02:26
· 164 commits to main since this release

Post-quantum readiness (PQ-01). The threat is harvest-now-decrypt-later:
traffic recorded today and decrypted once a quantum computer exists. TLS
already negotiated hybrid ML-KEM by default on every hop; WireGuard did not,
because the protocol has no hybrid mode. Both WireGuard planes now mix a
preshared key into every handshake, a host can be pinned to post-quantum-only
TLS, and regressions are caught in CI and at release. Full detail, verification
commands and rollout order in docs/POST_QUANTUM.md.

Security

  • An mTLS-enforced host was still reachable in the clear on port 80.
    Client-certificate enforcement lives in the TLS connection policy, so it
    applies only once a request arrives over TLS; the plaintext listener served
    the same host with no check at all. force_https was a separate stored flag
    that nothing set when enforcement was turned on. The route builder now
    derives it from require_client_cert, which also covers every row already in
    the database, and strict_sni_host is pinned explicitly so a request cannot
    reach the host under a different SNI. Migration 00145 backfills
    force_https = 1 on existing enforced routes.

  • A host could require client certificates while serving without TLS. The
    client-authentication policy is part of the route's TLS connection policy,
    so with SSL off nothing is emitted: the host answered every request with a
    permanent 503 under the default fail-closed mode, or served everyone with no
    client-cert check at all under fail-open - while the panel listed it as
    locked. All three ways into that state are now refused: creating a host
    (checked after the plan gate settles SSL, so a plan without SSL cannot force
    it), saving the host edit page (checked after the external-upstream branch,
    the last thing that can turn SSL back on), and PATCH /api/v1/routes/{id}
    clearing ssl_enabled on an enforced route. The add form and the edit page
    now state the dependency, and a host already in this state is shown as
    "mTLS saved - SSL off, not enforced" instead of "mTLS enforced".

  • "Require a client certificate" was silently discarded when adding a host.
    The create handler validated require_client_cert and the chosen CA, then
    never wrote either column, and the add-host form had no such inputs - so the
    check guarded a value that was thrown away. An operator who expected a new
    host to demand client certificates got one that accepted anyone, with no
    error and no warning. The add form now offers mTLS whenever an active CA
    exists, the flag and its trust anchor are persisted in the same INSERT as the
    route (so the host is enforced on its very first config push, never briefly
    open), and a create that asks for enforcement without a usable CA is refused
    instead of quietly downgraded. Existing hosts are unaffected.

  • Enabling mTLS on one host broke the TLS handshake for every other host on
    the same node.
    Caddy only supplies a default connection policy when the
    policy list is nil; with any per-SNI policy present, an unmatched
    ClientHello gets no server TLS configuration available and the handshake
    dies. One mTLS route therefore took down plain HTTPS for every other site on
    that node - and TLS-ALPN-01 renewals with it, so certificates quietly stopped
    renewing. The emitted policy list now always ends with a catch-all {}.
    Verified against a real Caddy 2.11.4 both before and after. No configuration
    change is needed; the fix lands on the next config push.

  • Rotating APP_SECRET orphaned two node secrets. cmd/rotate-secret was
    missing caddy_nodes.tunnel_privkey_e2 and caddy_nodes.admin_proxy_key_enc
    from its column list, so a rotation left the tunnel server key and the
    agent's admin-proxy key sealed under the old key - bricking WireGuard tunnels
    and authenticated config push on the next restart (security review CRYPTO-01,
    round two). Both are now rotated, along with the new preshared-key columns,
    and a column whose migration has not run yet is skipped with a log line
    instead of failing the run, so a newer image can rotate an older database.

Added

  • wstunnel bumped 10.5.5 -> 11.0.0 on both pins. The WSS tunnel binary is
    pinned twice - in the node-agent image and in the customer install script -
    and a bump that lands on one side only leaves half the tunnel behind, so a
    CI gate now fails when the two version or sha256 pins disagree. Upstream
    calls v11 non-breaking; verified locally with a real WireGuard handshake and
    traffic over WSS through Caddy, in all four client/server version
    combinations, plus --restrict-to still rejecting a non-WireGuard
    destination.

  • Preshared keys on the panel-node WireGuard mesh. The mesh carries Caddy
    config pushes, manual-certificate private keys included, so a recorded
    handshake is worth storing until X25519 falls. A node joining now gets a
    32-byte PresharedKey automatically, gated on the join script declaring
    support - a node bootstrapped from an older cached node-join.sh joins
    without one and keeps working. For nodes that joined earlier, the node detail
    page stages a key and prints a one-time command
    (curl -fsSL https://<panel>/install/node-psk.sh | sudo bash -s -- ...); the
    current key stays live until the node fetches, applies and confirms the new
    one, so the two sides are never more than one handshake apart. Confirm is
    idempotent for the token's 30-minute lifetime, so a lost response is retried
    rather than rolled back into a split mesh, and the staged key is cleared only
    once the panel has written its own WireGuard config - a repeat confirm can
    therefore never report success for a promotion that is still able to roll
    back. wg0.conf is replica-local and each replica feeds its own WireGuard
    sidecar, so every replica re-renders it from the database every 60 s, inside
    WireGuard's 120 s rekey window; that is what makes a rekey confirmed against
    one replica of a multi-replica panel safe. Clear PSK is the emergency exit
    and needs a matching edit on the node - and if the panel's own config cannot
    be rewritten, the clear is rolled back and reported as a failure instead of
    printing that node-side command, which on its own would take the mesh down.
    Note the timing: WireGuard holds an established session until the next
    handshake, so a mismatch takes up to ~120 s to appear, stops config pushes to
    that node, and does not interrupt traffic the node is already serving.

  • Preshared keys on customer tunnels. Per-peer keys on wg-tun0, gated on
    both sides: a peer only gets one when every node in its group reports
    support, and node-agent declares its support on the peer pull itself, so
    the panel knows what it is talking to before it decides what to hand over. An
    agent that declares nothing while the node still has PSK-bearing peers is
    refused with 409: it would apply the peer set without the keys and drop
    every one of those tunnels on the next wg syncconf, whereas the config it
    is already running keeps them up. A node with no PSK-bearing peers is served
    as before, so a downgraded fleet is not bricked. The capability is also
    re-asserted on every stats report and an absent field means "not supported",
    so rolling an agent back clears the flag; the 1 -> 0 edge writes a
    node.psk_capability_lost audit entry with the number of affected peers,
    whose configs then have to be re-downloaded. Key rotation degrades instead of
    failing: if a group node lags, the rotation proceeds without a PSK, clears
    psk_enc so the node's pull and the rendered config still agree, and audits
    wg_peer.psk_dropped. Existing peers stay classical until their next
    rotation or config re-download.

  • Post-quantum-only TLS per host. Opt-in on the host's SSL & Protocols tab;
    pins that hostname to curves=[x25519mlkem768] and TLS 1.3, so a client
    without hybrid ML-KEM fails the handshake instead of quietly negotiating
    classical X25519. Connection policies are merged per SNI, not per route,
    and the strictest wins - a TLS policy is matched before the request path is
    known, so enabling this (or mTLS) on one path enforces it for every route on
    that hostname, and two routes on one hostname with different mTLS CAs keep
    the first by route id and log the conflict. The policy is emitted only when
    SSL is on for the host and the node declares Caddy 2.10 or newer
    (caddy_nodes.caddy_version, operator-entered, not probed): an older Caddy
    rejects the entire /load on the unknown curve name. That gate is per node,
    so the host page checks every node serving the host - the anchor plus the
    route_node_assignments fan-out - and names the one holding it back instead
    of judging by the anchor alone. Saving a host pushes to that same set, and
    saving a node's capabilities (the Caddy version included) pushes that node:
    connection policies are rebuilt per node, so anything less left fan-out peers
    on the previous policy while the panel called PQ-only - or mTLS - enforced,
    and route-level drift detection cannot see a node-level difference. When
    either gate blocks the policy, the host page says so and the push logs it.
    Breaks old clients on purpose - Chrome < 131, Firefox < 132, Safari/iOS < 26,
    curl without OpenSSL 3.5+. TLS-ALPN-01 is disabled for such a host rather
    than left to fail silently at renewal time, so its certificate must be
    issuable over HTTP-01 (port 80 reachable) or DNS-01.

  • Post-quantum readiness card in Settings. Read-only checklist of where
    hybrid protection is actually in effect: panel runtime, per-node Caddy
    versions, mesh PSK coverage, customer-tunnel PSK coverage, PQ-only hosts, and
    what is still classical upstream. PQ-only hosts are counted as distinct
    SSL-enabled domains: several path routes can share one hostname, and a route
    with SSL off gets no TLS connection policy at all, so neither inflates the
    number. The node list gained a PSK / PSK pending pill.

  • CI guards and a release handshake smoke test. The build fails on a
    GODEBUG that disables ML-KEM, on CurvePreferences in non-test Go code, or
    on caddy:2.11.x pins that disagree across the deploy files, the docs or the
    README. At release, the freshly built edge image is started and probed by
    digest: a stock Go client must negotiate X25519MLKEM768 and a classical
    X25519-only client must still connect. The gate runs before edge, latest
    and the semver tags are created, so an image that fails it is unpullable
    rather than merely unsigned.

Fixed

  • The WireGuard mesh was dead on every fresh install. The wg named volume
    is populated from the image at first start, but Docker skips that copy - and
    the ownership that comes with it - when the image's directory is empty, so
    the volume stayed root-owned and the non-root panel process could not write
    wg0.conf. Nothing recovered on its own; the node never came up. The image
    now ships a placeholder in /app/wg and /app/data so the copy actually
    happens. Found by an end-to-end run against real kernel WireGuard, which no
    unit test could have caught.

  • Every wg syncconf on a node re-randomised its WireGuard source port and
    blackholed panel -> node traffic for up to 25 s.
    node-join.sh wrote no
    ListenPort into the node's [Interface], so the kernel picked a random
    source port - and a new one on every syncconf. The panel's peer blocks
    carry no Endpoint (it learns each node's from the incoming handshake), so
    after every syncconf it kept sending to the dead port until the node's next
    PersistentKeepalive re-taught it. An e2e run against real kernel WireGuard
    measured one node walking 54347 -> 51615 -> 38703 -> 46495 across three
    syncconfs, with the panel unable to reach it in between; node -> panel
    recovered on its own, which is why this went unnoticed. node-join.sh now
    pins ListenPort = 51820 (override with --wg-listen-port), and
    node-psk.sh adds the line to a config that lacks it, so an
    already-joined node heals on its first rekey. Predates preshared keys - it
    affected any syncconf on a node - but the PSK rekey is the first flow that
    runs one routinely. The panel's own sidecar was never affected: it has
    always rendered ListenPort.

  • Removing a mesh preshared key never took effect - both ends kept using it
    and the mesh split at the next restart.
    wg syncconf cannot remove a
    PresharedKey: a peer block without the line means "leave the current key
    alone", not "clear it". Every path that removes a key relied on it, so all
    three were silent no-ops. Clear PSK rewrote the panel's wg0.conf and
    reported success while the key stayed live on the interface; the node-side
    command it printed (and the one in the docs) did nothing either; and a
    rejected confirm during a first-time Enable PSK told the operator "the old
    key is back in place" while the node kept the staged key, dropped off the
    mesh ~180 s later and left nothing on either side to explain it. The sidecar
    now clears the key of any peer the rendered config no longer gives one,
    node-psk.sh clears the staged key when it rolls back to a config that has
    none (falling back to an interface restart, and failing loudly with the exact
    recovery command rather than claiming a rollback it could not perform), the
    node-agent does the same for customer peers whose key the panel dropped, and
    the panel's flash message and the docs now print
    wg set wg0 peer <panel-public-key> preshared-key /dev/null. Found by a new
    end-to-end harness (make e2e-psk) that drives real kernel WireGuard and now
    asserts the live interface, not the config file.

  • Upgrading a node-agent could clear the preshared key of every customer
    tunnel on that node.
    The peer pull negotiates preshared-key support on the
    request header, but only ever wrote the stored capability downwards: a node
    rolled back and then upgraded again pulled with X-HPG-Agent-PSK: 1 while
    caddy_nodes.agent_psk was still 0, so it was served a keyless peer set and
    read that as "the panel dropped every key". The pull now records the
    capability the request declares before it looks the peers up, and refuses
    with 500 if that write fails; and removal is no longer inferred from an
    omitted field at all - the answer carries psk_managed when the panel is
    really stating the whole key set, and the agent clears nothing without it.

  • A preshared-key removal the sidecar could not apply was reported as
    success and then never retried.
    The removal loop lost a failing wg show
    through a process substitution (whose exit status never reaches the caller)
    and turned a failing wg set into a log line, so the reload always looked
    complete and the watch loop advanced its mtime marker. An unchanged render
    keeps that mtime forever, so the retry never came: the manager could keep
    serving a key the panel considered cleared until someone restarted the
    sidecar - and if the operator cleared it on the node side meanwhile, the
    control-plane mesh split. Both failures now abort the reload, the marker
    only moves once syncconf and every removal succeeded (so the next 10 s tick
    retries), and a key that is still live is logged as such by peer public key
    on every tick. make e2e-psk injects both faults against real WireGuard.

  • The WireGuard sidecar was told to wg syncconf on every unrelated admin
    action.
    The rendered wg0.conf carried a generated-at timestamp in its
    header, so every render produced a different file and the sidecar, which
    reloads on a change to it, reloaded the mesh interface each time. A render now
    compares the part of the file that actually configures the interface and skips
    the write when it is unchanged. Affects every multi-node install, post-quantum
    or not.

  • docs/INSTALL.md still pinned caddy:2.11.3 for the lite stack while
    the rest of the repository moved to 2.11.4.

Migrations 00142 (mesh PSK columns), 00143 (customer_wg_peer.psk_enc,
caddy_nodes.agent_psk) and 00144 (routes.tls_pq_only). All additive and
defaulting to the previous behaviour.