Skip to content

v2.18.1

Choose a tag to compare

@AltanS AltanS released this 11 Nov 13:09
· 48 commits to main since this release

A HALF project stays HALF until every box it waits for got a deploy. A later run that skips such a box no longer clears it.

Fixed

  • A bay up that failed on one box left the project HALF, but the next run cleared it even when that run did not deploy the failed box. The failed run commits its compile, so the next plan of the project often has a smaller scope: no build change (no build server), no route change (no ingress box), no move (no old box), no planned step of another project. That run passed on the project's own box and wrote result: ok. The fleet then said the project was fine, while the failed box never got the deploy. bay rollback and bay up --take-over cleared it the same way.
  • A run with no host map at receipt time took the env-wide result for every box of the box env, also for the boxes its scope skipped. Only the boxes of the scope get a result now.
  • With traefik_split_entrypoints: true, a cold boot could leave Traefik down. Traefik binds websecure_tailnet to the tailnet address, and tailscaled has no address until it reaches Headscale. Traefik exited with listen tcp <ip>:443: bind: cannot assign requested address, which stopped the public 80 and 443 too. On the Headscale host it was a deadlock, because Traefik serves Headscale. The traefik role now sets net.ipv4.ip_nonlocal_bind=1 (live, and in /etc/sysctl.d/99-bay-traefik-nonlocal-bind.conf) before it writes the Traefik config. The listener still binds only the tailnet address. The variable traefik_nonlocal_bind follows traefik_split_entrypoints; set it to false to opt out (the file is removed, the live value stays until the next reboot). IPv4 only.
  • The boot unit bay-infra-boot.service (bay-infra-ensure.sh) removed Traefik when port 80 was not bound 30s after boot, and made a new one from the infra compose file. The removed container came from the Bay reconciler (label com.bay.config-hash). The compose copy has no such label and may differ from the Bay spec, so the next rig deploy planned a Traefik recreate ("the container has no config-hash label"). The recreate could not help anyway when the cause was a missing tailnet address. The script now never removes or replaces a container with that label. A labelled container that is dead or restarting, or Traefik with port 80 unbound, gets one docker restart, then a WARNING in the journal that names the likely cause. An exited container stays stopped, so a deliberate stop survives a boot. Compose create stays for a container that does not exist, and the log says the next bay deploy --rig <env> recreates it under the reconciler. A container without the label keeps the old behaviour.

Changed

  • A pending or failed lock record has a new key half_boxes: the boxes that still need a deploy. bay up writes it with the pending record, before it touches a box. After the deploy it keeps the boxes that failed or wrote no receipt from this deploy. A one-project run counts every box of its scope; a run of the whole environment counts the project's own box and the old box of its move.
  • A later run removes a box from the list only when its scope covered the box and the box passed. The record stays failed while one box is left. Such a run exits 0 and adds a note: <project> <env> stays HALF: this run did not deploy box <box>, ... Run bay up <env> --project <project> --all-boxes. pinned rows of a HALF record carry half_boxes.
  • bay show ends the HALF reason with the boxes and the command that covers them (--all-boxes when a box other than the project's own box is left). bay show --json has envs[].half_boxes and envs[].next.
  • bay plan adds a note when its scope skips a box that the project still waits for.
  • Docs: docs/plan.md has a new section "A HALF record names its boxes", and the HALF row of the bay show table names the boxes.

Upgrade notes

  • Every machine that deploys a fleet needs 2.18.1. An older CLI refuses half_boxes in a lock as an unknown key, so its compile, plan and deploy stop with a lock schema error. bay.compat.json stops it earlier: the first 2.18.1 bay up writes deployed_with: 2.18.1 in the same claim commit that first writes half_boxes, so a 2.16.0 to 2.18.0 machine stops with bay_older (exit 45) before it writes anything. lock_version stays 2.
  • A record that is HALF before the upgrade has no list, so it waits for every box of its box env. A one-project run no longer clears it. Run bay up <env> in the fleet (the whole environment), or bay up <env> --project <name> --all-boxes, once per such project. bay show names the command.
  • Split-entrypoint hosts: run bay deploy <env> --tags traefik on every such host. The sysctl takes effect at once and recreates no container. This tag restarts Traefik only when the rendered Traefik config changes, and this release does not change it, so Traefik keeps running. The fix matters at the next cold boot.
  • Every rig host: run bay deploy <env> --tags boot_safety (--tags turns the rig roles on) to install the new /usr/local/bin/bay-infra-ensure.sh. The boot_safety role runs from deploy.yml (rig roles only), not from provision.yml. It only rewrites the script. The unit file does not change and no container restarts.