v2.0.0

ZeroFS vNEXT: a new storage engine

This is the largest change ZeroFS has shipped. File contents no longer live inside the LSM tree. They are now written directly to the object store as immutable segment objects: every 32 KiB extent is compressed, encrypted, and packed as a frame into segments of up to 256 MiB. The LSM tree keeps only metadata: inodes, directory entries, and one 32-byte pointer per extent.

The result is a much, much leaner system: less CPU, less memory, far fewer S3 requests, and a fraction of the network traffic. Data now moves between ZeroFS and the object store the way a native S3 client would move it: large multipart uploads on write, large ranged GETs on read, each byte compressed and encrypted exactly once so throughput approaches raw S3 performance. Several operational features disappear because the problems they existed to manage no longer exist, and the practical size ceiling moves: multi-petabyte volumes are now easily within reach.

Performance

• Multi-petabyte scale is now easily within reach. Per-extent metadata is 32 bytes per 32 KiB of data (about a thousandth) so a petabyte of file data carries on the order of a terabyte of metadata, and that metadata is the only thing compaction still rewrites. Garbage collection scans one counter per segment (a petabyte is ~4 million segments), and segment keys shard across 256 prefixes so a large volume never hot-spots a single key prefix at the provider. The on-disk format itself tops out at 16 EiB.
• Major write throughput improvements.
• Major read performance improvements. A point read is one ranged GET of exactly the frame's bytes. Multi-extent reads coalesce every contiguous run within a segment into a single GET and decode the frames in parallel. Two prefetch layers (logical read-ahead across segments, plus adaptive per-object windows ramping to 8 MiB, kept 4 deep) keep sequential streams ahead of the reader. Segment packing re-sorts a file's live extents into physical order, so a whole-file read of a packed file can be a single GET.
• Major reduction in network traffic, because file data no longer rides compaction. Previously, LSM compaction repeatedly downloaded and re-uploaded file contents as SSTs merged. Segments are immutable, dense full-size segments are never rewritten, and compaction now only touches the small metadata SSTs. Deployments no longer need to provision large network throughput for background churn: sustained bandwidth now tracks the rate data is actually read and written, not a compaction multiple of it.
• Major reduction in requests issued to S3. Writes batch into large segment uploads instead of per-fsync WAL churn. Reads coalesce and prefetch. Garbage collection scans per-segment live-byte counters (O(segments), not O(extents)) and reads segment directories with two ranged GETs instead of whole objects.
• Major memory and CPU improvements. File data no longer flows through memtables, block caches, bloom filters, or compaction. Each extent is compressed and encrypted once when written, not re-encoded every time compaction rewrites an SST.

Caching

• The cache stores raw object bytes: compressed and encrypted. The same cache disk now holds roughly a compression-ratio multiple of the logical data it held before. It also means file data on the local disk is never in plaintext anymore; only decoded metadata blocks are cached in the clear.
• The cache is populated on writes. Sealed segments enter the cache from the bytes in hand, and single PUTs write through, so a read after a write needs no GET.
• With a disk cache sized to your working set, ZeroFS barely touches the object store. Reads are served locally; the object store sees sealed segment uploads and periodic metadata flushes.
• memory_size_gb now budgets both caches (the raw-parts cache previously took a fixed 128 MiB outside the budget), and a new warm_metadata key controls metadata warmup at startup and after compactions.

Durability

• The WAL is gone, permanently. fsync durability now comes from a strict flush barrier: seal and upload the open segment, then flush metadata. A durable manifest can never reference a segment that wasn't uploaded. This invariant is exercised by failpoint crash tests and the Jepsen suites in CI.
• The separate-WAL object store feature is gone with it: fsync latency no longer depends on a low-latency side store.
• Transient object-store errors on the data plane are retried instead of surfacing a single flaky response as EIO.

High availability

• Frames still in leader RAM ship with the replicated operation, and a promoted standby rebuilds any missing segment objects at takeover the acknowledged-but-unflushed write guarantee carries over to the new engine.

Operations

• The standalone compactor is gone. zerofs compactor and --no-compactor no longer exist. Metadata compaction always runs inside zerofs run; file data is reclaimed by the in-process segment garbage collector. Remove old compactor service units.
• The data plane has no tuning knobs; segment size, prefetch, and GC parameters are fixed constants, documented on the new Storage Engine page.
• Metadata compaction logs are digested into one line per interval with an L0 backlog warning; metrics moved from the slatedb_ prefix to lsm_.

Breaking changes

• New volumes only. This release writes and requires the zerofs-meta-extent-v1 layout. Volumes created by earlier releases are refused at open with a clear error. There is no migration tool; older volumes remain readable and writable by the release that created them.
• [lsm] wal_enabled and `[lsm] max_unflushed_gb. Neither has a replacement.
• zerofs debug list-keys prints extent_index where earlier releases printed chunk_index.
• Prebuilt release targets are now Linux (amd64, arm64), macOS (x86_64, aarch64), and FreeBSD (amd64).