Keepword v2.1.0

Fixes from an end-to-end bug hunt, most of them for what a hostile page or
peer could make a witness do, and a logo. 2.0.0 and 2.1.0 witnesses
federate and verify each other's bundles, all but re-running the other
version's normalizer. The normalizer is new, though, and captures only
compare when they were normalized the same way: until every witness runs
2.1.0, verdicts and rechecks draw on fewer witnesses, and a witness
doesn't compare a URL's first capture after the upgrade with the one
before. A TLSNotary session needs both witnesses on 2.1.0. Upgrade every
witness.

Added

• A logo: a gold seal stamped with a quotation mark, for what a page
  said, sealed. docs/assets has the seal on its own (logo.svg, and
  logo-512.png for avatars) and with the name, for light and dark
  backgrounds (logo-wordmark.svg, logo-wordmark-white.svg). The
  banner and the social preview use it (social-preview.svg is the
  source of social-preview.png), and the web UI shows it in its header
  and as its tab icon.

Changed

• Normalized text is keepword-norm/3. The normalizer has limits for
  hostile pages now, a link's text leaves out the links nested in it and
  only HTML <a> elements are links (see Fixed), and where tags are
  misnested, a link's text no longer runs on past the link or stops
  short. Such pages normalize differently, and as every normalized text
  starts with the version, every normalized hash changes.

Fixed

• A hostile page could crash a witness or keep it busy for days. The
  normalizer read pages recursively, so about 25 kB of nested tags
  overflowed its stack and aborted the node. The HTML parser took time in
  the square of how deep a page nests (minutes for a megabyte, days for
  the 32 MiB a capture may have), formatting tags it reopens in every
  paragraph made 160 kB of markup take 3.6 GB, and a link nested in links
  repeated the text of every link inside it: a 1 MB page normalized to
  4 GB. Pages are now read without recursion, and tags nested more than
  512 deep are ignored, their text kept; in SVG and MathML, <link>,
  <style> and the like open elements too, and there they close at once.
  A page's tree stops growing at one node per two bytes, a link's text
  leaves out the links inside it, and only HTML <a> elements are links:
  an SVG or MathML <main> can be where the normalizer starts, and <a>
  nests freely there. A page built for it still takes about a minute.
• Diffs of big rewrites could run for days. Finding the fewest changes
  between two versions takes time in the square of the lines changed, two
  minutes for 100 000. After a second, the rest now shows as replaced,
  and a line a diff shows as both removed and added, such as an update
  notice the page already had, no longer makes an edit disclosed.
• Gossip from newer versions could stop a node syncing with a peer.
  Pulls skip message kinds a node doesn't know, but a page of nothing
  else didn't move the cursor past them, so the node asked for that page
  forever. Pushes carrying them failed authentication, so their sender got
  no observation receipt. Pulls now move past every message, and pushes
  list the IDs of the messages they carry (2.0.0 witnesses ignore them).
• Watch requests for the longest intervals were due every round. The
  network takes requests for any interval from ten minutes up. Past
  292 million years, the interval wrapped around in the scheduler, and
  every witness assigned captured the URL every 15 seconds (debug builds
  panicked). Such a watch is now never due again.
• keepword log audit panicked when leaves were missing under a tree
  head, which is what it is there to find, and took time in the square of
  the log's size. It also ended with "VERIFICATION FAILED" when every
  check passed; it now says whether the audit passed.
• One bad calendar or explorer answer stopped anchor upgrades. A
  calendar answering anything but a timestamp failed every upgrade, and a
  failed block lookup discarded the proofs calendars had just completed.
  Each calendar or explorer that fails is now logged, once a round, and
  the rest go ahead.
• keepword request withdrew the old request before checking the new
  one, so a replacement the network refuses, such as --every 1m, left
  the URL with no request at all.
• keepword init wrote configs it can't load, for instance with
  --country DEU. No command worked after that, not even keepword config
  or keepword init. It now refuses them and writes nothing.
• keepword config unset didn't turn off drand or the block
  explorer. Left out of the file, beacon.drand_url and
  anchor.esplora_url loaded as their defaults again, so the node kept
  using api.drand.sh and blockstream.info, and tests meant to run offline
  fetched drand beacons. Unset, they are now saved as "", which turns
  them off. One unset with 2.0.0 is still missing from the file: unset it
  again.
• A failed round of watches stopped keepword serve, API and gossip
  included, for instance when the database stayed busy for ten seconds.
  It is now logged and retried, like a failed sync.
• A capture that couldn't be compared with the one before was
  "unchanged" in keepword capture and the watch log, after new site
  rules for instance. They now say it wasn't compared.
• A bundle's TLSNotary receipt could be dated 292 million years from the
  capture and pass the check that both happened within ten minutes: the
  subtraction wrapped around. Debug builds panicked on such bundles, and on
  pushes sent at such times.
• A TLSNotary session could hang for good. One in six stalled
  locally while the two witnesses set up MPC-TLS, and one CI run in
  twenty. The stall was inside TLSNotary, which has since fixed its stream
  multiplexer and replaced its executor; keepword-tlsn now builds against
  that revision, where none of 800 sessions hung. Sessions there share
  one thread pool, where each used to start a thread per CPU core.
  The revision speaks a newer protocol: a 2.0.0 prover hangs on a 2.1.0
  verifier, and a 2.1.0 prover gets an error from a 2.0.0 one.
• Notarization had no timeouts. A verifier that went quiet left
  keepword-tlsn capture waiting forever, and a prover that went quiet,
  or never sent its hello, held a verifier's connection for as long as it
  liked. Either side now gives up on a session after ten minutes, and on a
  hello or a receipt lookup after 30 seconds. A session that fails or is
  refused now closes its connection, and a refused prover still hears why.
• A flood of connections stopped keepword-tlsn serve. Once it ran out
  of file descriptors, at about a thousand open connections with the usual
  limit of 1 024, accepting failed and the service exited. It now logs the
  error and tries again a second later, and idle connections close within
  30 seconds.

Install

On a server, the installer builds from source and sets up the service:

git clone https://github.com/keepword-net/keepword.git && cd keepword
sudo sh scripts/install.sh --domain witness.example.org --caddy

The binaries below are static Linux builds of the keepword command
(without headless rendering). Check them against SHA256SUMS.