Storage node v0.23.0+ds.17

Storage node v0.23.0+ds.17 for Linux x86_64 (Ubuntu 24.04 or newer).

Running nodes install it by themselves within the staged-rollout window.
The archive is signed with ML-DSA-65; check it with:
ant-keygen verify --key release-signing-key.pub --input ant-node-cli-linux-x64.tar.gz --signature ant-node-cli-linux-x64.tar.gz.sig

The archive also contains LICENSE-MIT and LICENSE-APACHE, plus THIRD-PARTY-NOTICES.txt and RUST-STD-COPYRIGHT.html with the licences and copyright notices of the third-party code in the binary and of the tools used to build it.

What changes

ds.17 nodes work next to ds.16 and older nodes. They send two new kinds of message, sync pages and the full notice, only to peers whose user agent names ds.17 or later; every other peer is sent what it was sent before. Prices don't change, but a node at the full line no longer quotes for new data.

Neighbour sync is paged

• The limit it lifts. A node's hints for a neighbour (the keys the neighbour should hold, and the paid-list keys it should track) went in one message of at most 10 MiB, about 327,000 keys. A node holding more could no longer build that message, and its sync stopped.
• Pages. Between ds.17 nodes, each side's hints now go in pages of up to 8,192 keys (256 KiB), replica hints first. Each page is its own request and answer, the next sent only once the last is answered, up to 256 pages a side, and one exchange at a time with each neighbour.
• Windows. Hints that come to more than one exchange carries go a window at a time, each exchange carrying the next keys of each list. A window moves on by the keys that actually arrived, so an exchange that ended part way is picked up in the next one.
• Busy, not faulty. A page the other side can't take now is answered busy, and the requester tries again in a later cycle. Pages carry no trust reporting: silence, a timeout or a busy answer never counts against the neighbour.
• Bounds. Each page waits at most 30 seconds, an exchange runs at most 2 minutes, and a node holds at most 64 MiB of a neighbour's keys across the exchanges it has asked for.
• Older peers (ds.16 and before, or a peer whose release isn't known) get a single message, as before, which every release reads.
• Bootstrap. A node starting up tries again, after 5, 10 and 20 seconds, a neighbour that answered busy, and counts a neighbour as done only once its last page has arrived.
• Scans of a node's keys, for hints and for the hourly storage commitment, now work from one snapshot of the routing table instead of a lookup for every key.

Full, not faulty

• The falling floor. Below 10% free, a node may erase records far from it that its nearer peers prove they hold, farthest first, until 15% is free. The reach now grows as free space falls:

  Free space below	Records may be erased beyond
  10%	10 times the distance to the node's 7th-nearest peer
  7.5%	5 times that distance
  5%	2 times that distance
  2.5%	the edge of the node's own range

  Each step also fires once free space is down to the 1 GiB full line plus 4 MiB, so on a small disk (under about 40 GiB) every step runs before the node is full. Nothing inside a node's own range is ever erased, and every erasure keeps its possession check. ds.16 nodes stop at 10 times, and never erase while the network is small.

• Free inodes count as free space. The smaller share of free bytes and free inodes drives the steps above, and below 2.5% free inodes a node is at the full line.

• Erasure in two phases. A record chosen for erasure is first left out of every sync hint, and erased only in a pass a whole sync cycle later. A record leaves a node's range only after 3 days out of it, and that wait now survives restarts (out_of_range.mdb in the node's data folder).

• Full. A node at the full line with nothing left it may erase is full. It quotes and takes no new data, but keeps serving and proving what it holds, still answers a chunk it holds as already stored, and still takes updates of the pointers it holds. It leaves "full" once 256 MiB above the line is free again.

• The full notice. A full node tells its ds.17 peers so, renewing the notice every 10 minutes. They send its records to the next nearest node instead, widen their windows around it (at most 20 nodes), and don't charge it for a record it can't take, in one bounded case: its notice is under 40 minutes old, it answered that it lacks the record, the check isn't pinned to its storage commitment, it was never seen proving the record, and, after a write, it wasn't paid for it. Nodes keep the record of keys peers were seen proving across restarts (seen_holders.bin).

• Quotes at the line. A node at the full line, or full, no longer quotes for new data, so no client pays it for a record it couldn't keep.

• ds.16 and older nodes don't read the notice, and keep charging a full node until they update.

• Logs. A node logs Node full: since ... on entering "full", Node full: no longer on leaving it, and Node full: not at start when it restarts with room. These lines are kept at the full line, as warnings are. The hourly prune line adds floor_step=, being_erased=, near_full_peers=, few_holders=, seen_fill=, seen_false= and full=.

• Filesystem. For a disk of its own, XFS, or ext4 made with -i 4096, keeps inodes from running out before bytes.

Direct addresses before relays

• A node now dials a peer's verified direct address before its relay. Before, traffic to a reachable node could pass through another node's relay.
• The order is: a LAN address when the peer is behind the node's own public IP address; a verified direct address; the peer's relay (still skipped when the relay is behind the node's own public IP address); an unverified address; and a LAN address only when nothing else is known.
• A node behind NAT has no verified direct address, so its relay is still dialled first, with no direct attempt spent before it.
• The relay switch (ANT_RELAY_SERVING) is unchanged.
• The change is a patch to saorsa-core 0.29.1, vendored in vendor/saorsa-core, whose own tests now run in the Test workflow.

Updating the image (Docker)

Auto-upgrade replaces only each node's binary. The rest of the container changes only when you pull a new image: docker compose pull, then docker compose up -d. ds.17's image brings:

• full nodes in nodes status: docker exec storage-node nodes status lists each node that is full, since when, and whether bytes or inodes put it there.

Without the pull, the nodes still install ds.17 by themselves, with everything above that is part of the node: paged sync, the falling floor, "full, not faulty" and direct dialling. Only the nodes status list of full nodes needs the new image; without it, look for Node full: in the node logs (docker compose logs | grep 'Node full:').

What ds.16's notes say about images older than the one that ships darwin-nodes still applies: on such an image, pull the new one.