v2.4.4

HexDB 2.4.4 fails over in about 15 to 20 seconds instead of half a minute, adds an IP blocklist and per-tessellation file quotas, signs the answers a replica passes on, and makes small documents several times smaller on disk and aggregations and reads faster.

Added

• IP blocklist. Administrators refuse an IPv4 or IPv6 address, or a CIDR range, on every hex: POST /blocklist {"address": "203.0.113.0/24", "note": "...", "expires_in_hours": 24}, GET /blocklist, DELETE /blocklist/{id}, or the admin UI's Blocklist page. A blocked client gets 403 blocked before authentication (trusted proxies apply when finding its address). Entries replicate, can expire, and are audited (security.block, security.unblock). /health, /ready and hex-to-hex routes are never blocked, and an entry covering your own address is refused. blocklist is now a reserved tessellation name.
• Per-tessellation quotas. A files tessellation can have limits of its own, whoever uploads (administrators too): PUT /tessellations/{name}/quota {"mb": 500, "files": 10000}, null to remove them. Everything it holds counts, earlier versions and trashed files included; past a limit uploads get 507 quota_exceeded. GET /tessellations/{name} shows quota and quota_used. Counting a user's quota now uses an index on uploaded_by, created the first time, instead of a scan.

Changed

• Failover in about 15 to 20 seconds. Discovery runs every 5 seconds by default (was 10), and a lost Overseer is replaced after three missed rounds. A peer isn't counted lost while another hex that answered this round still sees it, so a replica that's starved of CPU or cut off on its own can't elect itself and split the lattice. Lattices that set network.discovery_interval_seconds keep their value.
• Smaller files on disk (SSTable format 6). Small JSON documents compressed one at a time barely shrink. A file of at least 64 documents averaging under 8 KB now gets a zstd dictionary trained from its own documents, stored encrypted in the file, and used only when it saves more than twice its size. Order-like documents of about 250 bytes went from 177 bytes each to 40. Older files are still read, and compaction rewrites them in the new format.
• Faster reads. A stored document is decoded straight into typed fields without building an intermediate JSON tree: about half the work (a 5 MB document: 199 to 95 ms). Every read benefits, including queries and aggregations that read from disk.
• Faster aggregations. Documents are read in batches and parsed on every core, the document cache shares documents instead of copying them, and only the fields an aggregation uses are converted. A GROUP BY over 100,000 documents: 320 to 210 ms warm, 690 to 580 ms cold.
• Signed forwarded answers. The Overseer signs its JSON answers to requests a replica forwards (writes, and a sliced replica's reads). The replica checks the signature before passing an answer on and answers 502 if it doesn't verify, so an answer changed in transit between hexes is refused. Downloads and live streams still pass through as they come and rely on TLS between hexes. During a rolling upgrade, a replica insists on a signature only from an Overseer that announces it signs.
• Hidden deletes in the change feed. A reader whose role has a row filter now sees a delete event only if the deleted document was within the filter, so delete events no longer reveal that hidden documents existed. Readers without a row filter see every delete, as before.
• Sealed fields in memory. When a document a request read is freed, the decrypted values of its sealed (masked) fields are overwritten. This only runs while some role masks a field.

Fixed

• Hexes started together start with one Overseer. Hexes started at the same moment (a new lattice, or one restarted whole) could each start as Overseer until the next discovery round. A hex with network.peers set that finds no one now looks once more after its storage is open and it answers probes itself, so at least one of them finds the other. A hex started alone with peers configured takes at most 3 more seconds to start.
• Keys used only on replicas during a lattice-secret rotation. An API key verified with a previous lattice key is re-hashed with the current one, but only the Overseer could store the new hash, so a key used only for reads on replicas stopped working once the old secret was removed. A replica now asks the Overseer to re-hash it.
• Same-origin checks behind a router. When a router serves a hex under a public name (as hosted lattices are), the cookie CSRF check, CORS and live streams compared the browser's origin with the hex's internal host. The X-Forwarded-Host a trusted proxy sends now counts; from anyone else the header is dropped.

Upgrading

Restart the hexes one at a time, as usual: replicas first, the Overseer last.

SSTable format 6 can't be read by earlier versions. Once a 2.4.4 hex flushes or compacts, its files are in the new format, so going back to 2.4.3 or earlier means restoring a backup taken before the upgrade. Take one first if you might need to downgrade.

Full Changelog: v2.4.3...v2.4.4