Shepherd 1.0.0

The first stable release, and with it the promise 1.0.0 is for: shepherd.toml and the portal API
will not break under you within 1.x. A key you set today keeps meaning what it means; anything that
would change that waits for 2.0.0 and is written down here first.

What made this the release rather than another 0.x: a new install no longer starts in a text editor,
releases are signed so a running Shepherd can actually apply an update, per-device bandwidth comes
from the router rather than from steering traffic, and the one feature that cannot yet be trusted —
DHCP mode — says so on every surface and is off by default.

Added

• First-run setup in a browser. A new install no longer starts in a text editor. Start Shepherd
  with no configuration file and it brings up a setup page, prints the addresses to open (with a
  one-time token, since the page listens on every address so it can be reached from the machine you
  are actually sitting at), asks what it has to, writes a commented shepherd.toml and starts on it
  in the same process. shepherd setup runs it again later. It listens on 8788 rather than the
  portal's 8787, because it keeps serving while Shepherd starts.

  What it asks is deliberately short, and two of the answers it works out rather than asking for.
  The data directory is data beside the configuration file, created and checked while the page
  loads. Upstream resolvers are measured, not guessed: every candidate is asked the same
  uncached question several times from this host, and the recommendation prefers an encrypted (DoT)
  resolver that also blocks known-malicious names — but only while it answers every probe and stays
  under 250ms from here, so a flaky or slow one loses to a plain resolver that works. The numbers
  are shown, so the recommendation can be checked. The address question is still the reason the
  whole thing exists: every address with its interface, all ticked, because a device asks one
  specific address and a host serving only its Wi-Fi address while the router hands out the wired
  one is a network with no DNS, with nothing in the configuration looking wrong. A few catalog
  categories can be blocked network-wide from the page, switched on in the running policy the portal
  owns rather than copied into the configuration file.

  The page stays with you after the file is written, for the part that needs Shepherd running:
  it asks Shepherd's own listeners over UDP and TCP for a name that must resolve and one that must
  be blocked (the self-test from docs/DNS-QA.md, run on demand, not a second implementation), hands
  over the portal token concealed and asks you to confirm you have kept it, says what to set in your
  router's DHCP settings and to give the Shepherd host a fixed address, and offers the UniFi
  connection for per-device bandwidth. It does not reconfigure your router: the eero and UniFi APIs
  Shepherd speaks are the read-only ones, and neither vendor publishes a supported way to set a
  network's DNS server. Closing the page is the last step, and the listener stops there.

  Setup can also put the portal on HTTPS, which is what makes a passkey possible: WebAuthn binds
  a passkey to one hostname, so Face ID or Touch ID for the portal works at that address and nowhere
  else. The step is optional and skipped by leaving the hostname empty. Give it a name in a
  Cloudflare zone and an API token scoped to Zone → DNS → Edit on it, and setup checks that token
  against that zone before accepting it — the same read-only check the portal's Proxy tab makes — and
  refuses to finish with a hostname whose token has not been checked, because a token without edit
  rights is a certificate that never arrives. The token goes to <data_dir>/secrets.json (0600),
  never into shepherd.toml. Afterwards the page waits for the certificate, saying which of the slow
  parts it is on, then links to the portal at its own HTTPS name, where the passkey is registered on
  the Locking screen. Setup does not register it itself: that page is served over plain HTTP from a
  different origin, and a passkey belongs to the hostname's own.

  An existing shepherd.toml is kept as .bak; only the keys setup asked about are written, so
  every other default keeps following Shepherd's own; a held port is explained rather than refused;
  and a setup listener that cannot come up at all is logged while Shepherd starts on its defaults.
  It complements scripts/shepherd-doctor (the read-only host preflight) rather than repeating it.
  See docs/SETUP.md.

• ntfy is set up in the portal. Alarms → Notifications → Set up ntfy is now a real form — server,
  topic, access token or username and password, with Save, "send a test notification", a switch and
  "forget server" — instead of a shepherd.toml section to copy and a restart. Shepherd's standing
  rule is that no feature requires editing the configuration file, and the walk-through broke it.
  The server lives in <data_dir>/secrets.json (0600) beside the mail relay and the router
  connections (internal/secrets), is write-only like every other secret there (a saved token is
  never sent back; a blank field keeps the stored one, and only for the same server, so a credential
  can never be pointed at somewhere it was not entered for), and reaches the running delivery Router
  through the new Router.SetNtfy, so nothing about ntfy needs a restart. Saving publishes one
  notification as the check, which is also what the test button does: ntfy offers no way to ask
  whether a server and token would work without sending something, and a refused server is reported
  with nothing stored. [alarms.ntfy] in shepherd.toml is still read once and imported, so an
  install already using it keeps working; after that the portal owns it and a startup line says so.
  See docs/ALARMS.md.

• Releases are signed. The Ed25519 release key pair exists now: the public half is built into
  Shepherd (update.ReleasePublicKeyB64) and the private half is the SHEPHERD_SIGNING_KEY
  repository secret, so both workflows write SHA256SUMS.sig beside the checksums and a running
  Shepherd verifies it before applying an update. Until now every release was published unsigned and
  self-update could only report that a newer version existed. A binary built before this commit has
  no key built in and still refuses to apply anything; it has to be replaced by hand once.

Changed

• The lock screen leads with one way in, and tries a passkey by itself. It used to show the PIN
  and token forms together, plus a passkey button where one was available, leaving the owner to pick
  between things only one of which they could usually use. It now asks the server what is actually
  available to this browser (GET /api/login/methods, which replaces GET /api/login/passkey) and
  expands the first of passkey, PIN, token that is; the rest become a quiet line of links. The
  token is never hidden, only collapsed — it is the way back in when a phone is lost or a PIN is
  forgotten. Where a passkey is the way in, the ceremony now starts without being asked: once, and
  only on a page that is actually being looked at, so a cancelled attempt leaves the button rather
  than prompting in a loop. The new endpoint tells a browser without a trusted-device cookie that no
  PIN is available, which is the truth for it and says nothing about whether anyone else has one.

Changed

• Watch mode is deprecated. Per-device bandwidth now comes from the router over its own API
  (eero and UniFi), which measures both directions instead of estimating downloads, puts nothing in
  the path of your traffic, changes no setting on your router, and works on mesh networks where
  one-sided ARP steering cannot be made reliable at all. Watch mode stays in the tree, off by
  default, for the installs already running it, and gets no new features; the portal's Watch mode
  section is labelled deprecated and points at "Read from your router" above it, and starting with
  watch.enabled = true now logs the same. shepherd.example.toml said enabled = true while the
  code defaulted to false -- the example now matches the code. The reason it was given up on rather
  than merely switched off is written down in docs/WATCH.md: on an eero mesh the
  gateway reflects forwarded frames carrying a foreign source IP, so the first full-size segment of
  every flow never arrived, reproduced with both relays and with plain kernel forwarding. Source NAT
  stopped it dead (0/10 to 10/10 on the probe, zero reflected frames), so if this is ever revived it
  belongs in docs/GATEWAY.md as a DHCP/NAT mode, not as ARP steering.
• The Overview and Watch mode screens are gone. The Overview repeated what the Map, History and
  Bandwidth screens show better, so the portal opens on the Map and what only the Overview had (software
  updates, blocklist health, your own rules, the Private Relay switch) is the new System screen under
  Settings. Watch mode's status, device list and kill switch are the last section of Bandwidth.
• Bandwidth is mostly charts now. A live rolling chart with a bar pair per device, an Over time chart
  with round-number gridlines and a hover read-out, a share-of-traffic donut with a download against
  upload bar, and a trend line per busy device.

Added

• DHCP mode — ALPHA, off by default (docs/DHCP.md). Alpha means something
  specific and the doc says it plainly: no real device has ever taken an address from it. The lease
  exchange is tested end to end, but over loopback sockets on high ports, so the real port 67, the
  broadcast a client needs before it has an address, and a household's traffic crossing one network
  card twice have been reasoned about rather than measured. It is also the one thing here that can
  take a whole network off the internet. Nothing offers it to you: it is off in the default
  configuration, no first-run path mentions it, and the portal screen leads with all of this. Shepherd can serve the LAN's DHCP and
  masquerade the traffic of the devices it routes, so both directions of those devices' traffic pass
  through it -- downloads measured rather than estimated from acknowledgements, and QUIC no longer a
  blind spot. One NIC, one flat segment, the router still routing. [dhcp] mode has three positions
  rather than two: off opens no socket, observe opens a listener with no send path at all (it
  feeds the device registry from every DHCP packet on the segment and records what Shepherd would
  have answered, which is how the whole household's awkward devices get exercised before anything is
  at stake), and serve answers. Serving is gated on a typed confirmation in the portal and on no
  other DHCP server having replied on the segment for a settling period -- and it stops, with a
  critical alarm, if one appears later. Routing is a separate switch from serving, so
  mode = "serve" with route = false leases the whole house with the router's own address in
  option 3 and puts nothing on Shepherd's data path. The portal's new DHCP screen shows the leases,
  the addressing, what is blocking it, and address reservations you can add, edit and remove;
  each device gains its own routing opt-out beside its Watch switch, and a Reserve this address
  switch that pins it to the address it already holds. The LAN's range is yours to choose — the
  prefix, Shepherd's address, the router's address and the pool — and a chosen range is checked
  against the network as Shepherd can actually see it: this host's addresses, the gateway, and every
  address the device registry knows is in use. A range the router does not use yet is stored, not
  refused, with a list of what has to change first; serving stops until the list is empty, because
  Shepherd serves the leases and the router still routes. Failure is deliberately
  fail-open, departing from the fail-closed decision in docs/GATEWAY.md for a
  stated reason: an unhealthy Shepherd hands out the router's address instead of its own, so routed
  devices drift back on their own within one renewal interval without anything having to work.
• Every loop-brake trip is kept, with its raw evidence. last_loop_stop in watch.json now also
  records the relay's counts over the mirrored seconds, the echo-delay histogram (local, switch or
  wireless) and the sampled frames as hex, and each trip is appended to watch-stops.jsonl beside it
  so the next trip no longer erases the one before. Aimed at tracing why watch mode stops until it
  stops no more.
• UniFi as a bandwidth source. A UniFi Network controller keeps real byte counters per client,
  so unlike eero its figures are measurements rather than sampled estimates, and when both are
  connected the controller wins. Connecting one is a single form (address, a read-only account,
  site) with no verification code; the credentials are checked against the controller before they
  are stored. Both a UniFi OS console and a self-hosted Network Application work, discovered at
  login, and redirects are refused so a mistyped address cannot walk the session cookie somewhere
  else. See docs/TRAFFIC.md.
• Per-device bandwidth read from your router, for networks where Watch mode cannot run. A new
  internal/routerusage package reads what the router already knows and records it as the same
  per-device byte counts Watch mode produces, so the Bandwidth screen, the per-device chart and the
  history all work unchanged. Sources report either cumulative byte counters (exact) or
  instantaneous rates (integrated, and labelled as an estimate everywhere it reaches the owner).
  eero is the first source: connect it under "Read from your router" on the Bandwidth screen,
  which reuses the existing sign-in and keeps the session in secrets.json. It is unofficial --
  eero publishes no API for this -- and it reports rates only, so its totals are close rather than
  exact. Nothing is put in the path of your traffic, and no setting on your router is changed.
  See docs/TRAFFIC.md. The "Right now" card and the per-device rows are fed by
  the reader too while Watch mode is not measuring, so the live view is not left at zero; the
  rolling chart's caption says how often the router is actually read, since the line steps rather
  than curves. Measured against a known transfer, eero's sampling recorded about a sixth of it, so
  the screen and the docs call these totals a floor rather than a measurement. The live card shows
  only the eight busiest devices, since a router source lists every device it knows, and says how
  many it is leaving out. The kept sign-in also powers Device names: copy eero's names and
  profiles into Shepherd whenever you like, with no new verification code. It previews first, and
  names you set yourself are kept unless you explicitly ask for them to be replaced.
• ntfy setup walk-through under Notifications on the Alarms screen: an expandable section with the steps, a ready-to-paste [alarms.ntfy] block with a hard-to-guess topic suggested, and a line saying whether ntfy is on and where it publishes (never the token). GET /api/alarms/push now includes an ntfy object.
• A built-in DNS self-test ([selftest], on by default). Every minute Shepherd asks a known-good and a
  known-blocked name of every address it listens on, over UDP and TCP, so an address devices use with no
  listener behind it (the wired-versus-Wi-Fi failure that left a PS5 with "host not found") is caught within
  a minute instead of by a person. It also watches each upstream's error rate and latency over a rolling
  window, flags one above 5 percent errors or far slower than its peers, and warns when this host has a LAN
  address that is neither served nor listed in selftest.not_served. Results are a card on the Upstreams
  screen and GET /api/selftest; four new alarm types (dns-listener-down, dns-canary-failed,
  dns-lan-uncovered, dns-upstream-degraded) fire after two failing rounds and not again within the
  cooldown. The test's queries are kept out of the query log and device list. See
  docs/DNS-QA.md.
• scripts/dns-qa, an automated DNS QA suite (its tests run in CI with the rest): a real engine on real sockets in
  front of fake UDP, TCP, DoH and DoT upstreams, checking failover, hedging, stalled and blackholed upstreams,
  TCP fallback on truncation, the AAAA filter and ipv6hint stripping, blocklists and allowlist precedence,
  Private Relay NXDOMAIN, cache TTLs, and listener coverage on loopback aliases.
• dns.filter_aaaa ("auto" by default, or "on" / "off"). On a network whose devices have no
  IPv6 route to the internet (only private fd.. or link-local addresses) every AAAA answer is an
  unreachable address, and a device that tries it first or handles the failure badly reports a network
  problem. Shepherd now answers AAAA queries with an empty answer, and drops IPv6 hints from HTTPS
  records, while outbound IPv6 is unreachable. auto finds out by trying a TCP connect to three public
  IPv6 resolvers every minute: one success turns AAAA answers back on at once, two failed rounds in a
  row turn them off, and each change is logged. Filtered answers show as ipv6-filtered in the log.

Fixed

• The site was advertising a Shepherd that no longer existed. site/demo/, the live demo the
  marketing site serves, was hand-copied once and then drifted: fourteen files behind the portal, with
  no Bandwidth, Network health, Reverse proxy or vulnerabilities screen, and an index.html that had
  become a different, older page. The README and the site still showed the old HUD / WATCH / DETAILS
  navigation, the site called the installer "planned" and Watch mode "on by default", and both
  described the Bandwidth screen as built on Watch mode. The demo is now generated from the portal's
  own files by scripts/sync-site-demo, CI fails if the two drift again, the screenshots are
  recaptured from the current portal, and the copy says what Shepherd actually does: bandwidth read
  from your router, Watch mode deprecated, DHCP mode alpha, setup in a browser.

• The alarm engine's test-only tick hook could wedge the worker it runs on. The hook is called
  inline by the worker loop, so one that blocks stops the worker, and Close then waits on it for
  ever: a hung test binary rather than a failing test, which costs the whole suite its ten-minute
  timeout instead of naming the problem. The contract is now written down on the hook, and the one
  test that uses it signals without blocking. No change to the engine's behaviour -- nothing in the
  public API sets the hook.

• A list refresh started from the portal was untracked once its request returned. POST /api/lists/refresh answers 202 and refreshes in the background by design, but nothing held a
  handle on that goroutine, so nothing could wait for it to finish writing its list files. In the
  test suite it raced the temporary directory being removed and made CI fail intermittently
  (TempDir RemoveAll cleanup: directory not empty). The server now tracks background refreshes and
  can drain them. No change to how refreshing behaves for you: the request still returns at once.

• Filtered AAAA answers, and blocked names, could make a device query without end. A synthesized
  NODATA or NXDOMAIN carried no SOA in its authority section, and RFC 2308 negative caching needs
  one: with nothing to cache against, a client that prefers IPv6 just asks again. A solar inverter
  here asked for the same filtered AAAA forty-eight times a second, indefinitely, and never
  fell back to IPv4. Every synthesized negative answer now carries an SOA with a one-minute
  negative TTL, which matches how often the IPv6 probe runs.

• The Bandwidth charts overflowed the left edge of their card on a phone. An axis label wider than
  the gutter ("100 Mbit/s") was positioned from the right with no left bound, so it grew out of the
  card; it is now bounded on both sides, the gutter is wide enough to hold it, and the live, trend
  and totals grids no longer demand a minimum width wider than a phone's card.

• A failed update check caused by GitHub's anonymous rate limit (60 requests an hour per address, shared by everything on your network) now says so and when it resets, instead of "HTTP 403".

• Watch mode no longer eats duplicate TCP acknowledgements. The duplicate brake drops any frame it has already relayed inside 100 ms, but a receiver that sees a gap sends the same ACK again for every segment that follows, and TCP needs those to reach the sender (three start a fast retransmit; SACK recovery runs on them). One lost packet on the download path therefore cost the sender a full retransmission timeout, which shows up as stalled and failed requests. A repeated pure ACK is now relayed up to 8 times inside the window before it is treated as a loop, in both the kernel and userspace relays; data, SYN, FIN and RST frames are dropped on their first repeat as before.

• Unsteering now really restores a device. The corrective ARP reply that tells a device the gateway's real address again was sent with the gateway's MAC as its Ethernet source, a forged link-layer address that wired links and docks drop, so a wired device kept the claim and sent everything through Shepherd until its ARP entry expired. It now leaves from Shepherd's own MAC (the body still says the gateway's), which is accepted, and it is repeated four more times, half a second apart, in the background in case one is lost.

• Update checks, blocklist refreshes and the other feed downloads failed with a DNS timeout when the host's resolv.conf named resolvers Shepherd does not listen on (the usual case in a host-network container), so every list showed "Rejected". Shepherd's own fetches now resolve through the bootstrap servers.

• Subscribing to push notifications failed with "sw.js load failed": the service worker and web manifest were not embedded in the binary.

• The portal locked while it was in use. Idle locking counted only calls that mean a person is
  there, and looking at the map, panning the globe or scrolling made none, so a page in active use
  relocked after the idle window. The page now reports real input (at most once a minute) to
  POST /api/activity, which slides the window; a tab left open still locks on schedule. (A restart
  of Shepherd still ends every session, since sessions are held in memory.)

Added

• Bandwidth on the device card. The device panel now shows that device's last 24 hours of download
  and upload, from the same traffic history as the Bandwidth page.

Fixed

• The device panel's close button was hidden under the top bar. The panel started at the very top of
  the window, behind the LIVE bar, so its header and close button could not be clicked. It now starts
  below the bar.

Fixed

• A stalled upstream no longer makes DNS lookups fail. Upstreams were tried one after another with a
  two-second timeout each, so one that stalled (a slow DNS-over-TLS connection, or a resolver dropping
  packets) cost every query two seconds before the next was tried, longer than many clients wait.
  Shepherd now starts the next upstream after a short hedge delay (3x the first one's round trip,
  150 to 400 ms) or at once if the first fails, and answers with whichever replies first. An upstream
  that was hedged out still records its own success or failure, so a chronically slow one still falls
  down the ranking.