Repository navigation
v2.18.0
Headscale hardening: random IP allocation for new nodes, a consistent and checked DB backup, a locked-down container, and tighter file modes.
Changed
- New role variable
headscale_ip_allocationfeedsprefixes.allocationin the Headscale config. The default israndom. The value must berandomorsequential; theheadscalerole stops on any other value. Reason: ACLhosts:aliases and grants point at IPs, and withsequentialHeadscale gives the IP of a deleted node, and every grant on it, to the next node that joins. tailscale_registernow checks a host right after it registers: when its tailnet IP is notheadscale_server_tailnet_ip, the deploy stops and the message names the IP to set. Underrandom, a fresh control host no longer gets100.64.0.1, the default of that variable. A host that is already registered is never stopped by this check.- The Headscale backup no longer copies the live database with
docker cp. The DB runs in WAL mode, and a file copy can catchdb.sqliteanddb.sqlite-walat different moments. A root helper (/usr/local/sbin/bay-headscale-db-snapshot) runssqlite3 .backupon the live DB, runsPRAGMA integrity_checkon the copy, and fails unless it printsok. The snapshot and the noise and DERP private keys go to restic as before (headscale.tar, sameheadscale/layout). The live-waland-shmfiles are no longer in the archive. The temp copy is 0600 in the root-only/var/lib/bay-headscale-backupand is removed on every exit. A failed check stores no snapshot and sends the usualbackup.failedalert. The backup unit still runs as the app user and calls the helper through one sudoers rule (/etc/sudoers.d/bay-headscale-backup) that allows no arguments. Restic, retention, the size check, maintenance and alerts are unchanged.backup_headscale_source_pathis gone;backup_headscale_data_dir,backup_headscale_snapshot_binandbackup_headscale_snapshot_dirare new. - The Headscale container runs with a read-only root filesystem, a tmpfs for its unix socket (
/var/run/headscale, sodocker exec headscale headscale ...keeps working), the config folder mounted:ro,cap_drop: [ALL]pluscap_add: [DAC_OVERRIDE], andno-new-privileges. It still runs as root. DAC_OVERRIDE is the one capability it needs: the config and data folders belong to the app user, so root without it cannot readconfig.yamlor write the data folder (checked againstheadscale/headscale:0.29.7: with every capability dropped it does not find its config; with DAC_READ_SEARCH alone it cannot write its noise key). The reconciler learned theread_only,tmpfsandcap_addspec keys; a spec without them hashes as before, so no other container is recreated. - File modes:
/opt/headscale/datais 0700 and/opt/headscale/config0750 (both were 0755).db.sqlite, its-waland-shmfiles and every DB copy in the data folder or in/opt/headscale/backupsbecome 0600 (Headscale created them 0644). The role fixes the modes on every run and deletes nothing. SQLite creates new-waland-shmfiles with the mode ofdb.sqlite, so they stay 0600 after a restart. - The
headscalerole refusestailnet_identity_source: sqlite: the sidecar runs as a non-root user and can no longer read the DB. The defaultapisource is unaffected. - Docs:
docs/backups.mddescribes the snapshot helper, moves the old-waland-shmfiles away before a restore, and has a new "Restore test (Headscale)" section.docs/access-gateways.mdsays how to setheadscale_server_tailnet_ipunder random allocation.
Upgrade notes
- Run
bay deploy <env> --tags headscale,backupagainst the fleet that runs Headscale (both roles are in the deploy playbook, not in provision).headscalesets the file modes and recreates the container with the hardened spec.backupinstalls thesqlite3package on the host, the helper, its folder and the sudoers rule, and renders the new backup script. A fullbay deploy --rig <env>does the same. - The Headscale container restarts once (the spec hash changes). Nodes reconnect within seconds. Nothing else is recreated.
- A plain
bay upon that fleet also recreates the Headscale container, becausebay upbuilds the same spec.bay planshows it as aheadscalerecreate step. So the firstbay upafter the upgrade is not a no-op on the control host. Run the--tags headscale,backupdeploy first, so the modes, the helper and the container change in one step. - IP allocation: existing nodes keep their addresses; only nodes that join after the upgrade get a random free IP. Headscale reads every assigned IP from its database as used, and its startup backfill only fills a node that has no IP. To keep the old behaviour, set
headscale_ip_allocation: sequentialin group_vars before the deploy. - A fleet that sets
tailnet_identity_source: sqlitemust switch toapifirst, or theheadscalerole stops. - The first backup after the upgrade is a new-format archive. Restore older snapshots as before;
docs/backups.mdcovers both. - An old copy such as
/opt/headscale/backups/db.sqlite.pre-v0292-*becomes 0600 but stays. Delete it by hand when you no longer need it.