storage: report actual disk usage per VM and image - #21
Merged
Conversation
Ember reported provisioned sizes everywhere and actual sizes nowhere. The only command that tried, `ember debug storage-efficiency`, walked `.img` paths that exist only on macOS and reported zero on Linux. Add a `usage()` method to `StorageBackend` returning provisioned, exclusive, referenced, and logical bytes per volume plus pool totals. The method takes the state records and returns the whole set at once, so a backend that has to walk pool-wide metadata does it once rather than once per volume. ZFS answers from two `zfs` calls. dm-thin reads per-volume figures through a reserved metadata snapshot and `thin_ls`, which also covers volumes that are not currently activated. APFS reports `st_blocks`. Wire it to a new `ember storage usage`, a USED column on `vm list`, usage rows on `vm inspect`, and a pool line on `ember info`. Everything but `storage usage` treats measurement as best-effort, so listing VMs keeps working when the pool is unreachable. Removes `ember debug storage-efficiency` and the now-empty `debug` subcommand tree. See docs/STORAGE-USAGE-SPEC.md.
Follow-up to the accounting change, from two adversarial review passes. Semantics. `exclusive` was `usedbydataset + usedbysnapshots`, but snapshot-only space is by definition outside `referenced`, so both image rows shipped reporting more occupancy than they reference. It is now `usedbydataset`, a subset of `referenced` by ZFS's own definition, and the contract says plainly that this is occupancy and not what a destroy frees. The regression test now runs against the four rows a live pool produces rather than a fixture that zeroed the field which broke the invariant. Pool compression counted empty refreservation as perfectly compressed data, understating the ratio by 5%. `PoolUsage` gains `reserved`, the ratio divides by the occupied remainder, and the CLI prints the reservation as its own row so the volume rows visibly sum to the pool line. Read-only queries. dm-thin `usage()` no longer activates the pool, so `ember vm list` cannot load a pool table, attach loop devices, and run `thin_check` as a side effect of listing VMs. `try_usage` moved to `src/backend.rs`, breaking a `cli::vm` <-> `cli::storage` import cycle, and goes through a new fallible `try_create_storage` so a config naming an unimplemented backend no longer panics the three commands you would run to diagnose it. Labelling. `RATIO` is `COMPRESSION`, `Ratio:` is `Compression:`, the accessors are `compression_ratio()`, and `ember info` says `Capacity:` rather than a second row called `Pool`. Tests. All integration tests are `#[ignore]`d to match the rest of `tests/`, since the runner passes `--ignored` and `TestEnv` needs root. Fork creation was missing `--no-start` against a kernel that cannot boot. The unmeasurable-storage test is Linux-only, because the APFS backend reads neither field it clobbers. Adds the unit tests the spec asked for: derived quantities and their zero guards, metadata block scaling, and the thin-id join. Retargets the docs, README, and `tests/macos_storage.rs` that still referenced `ember debug storage-efficiency`.
The APFS usage model rested on st_blocks counting only the blocks a file does not share with a clone. It does not. st_blocks counts the blocks a file maps, so a fresh clone reports its origin's full figure while costing nothing, and summing it over an install of one image and fifteen clones overstates real occupancy by more than 5x. Replace it with a physical extent scan. fcntl(F_LOG2PHYS_EXT) gives the byte ranges a file maps, so sweeping every .img in the tree at once yields exclusive, shared and a true pool figure. That restores all four columns on macOS rather than blanking them, and it keeps the invariant exclusive <= referenced true by construction. Also record why the trait batches: on APFS a volume's exclusive figure is only defined relative to every other volume that might share its blocks, so a per-volume call could not express the answer at all. Correct the same claim where it is repeated in MACOS-SPEC.md, and flag it as unverified for reflinks in BTRFS-SPEC.md.
The APFS backend reported st_blocks as the exclusive figure. st_blocks counts the blocks a file maps, not the ones it owns, so a fresh clone reported its origin's full figure while costing nothing. A pure fork that consumed 13 MiB was reported as 285 MiB, eight of them as 2283 MiB against 10 MiB real, and pool.allocated grew every time a free clone was made, which also made capacity (allocated + available) drift upward on a volume that had not changed size. Read physical extents instead. fcntl(F_LOG2PHYS_EXT) gives the byte ranges a file maps and SEEK_DATA skips holes, so sweeping every .img in the installation at once splits the tree into what each volume holds alone and what the whole set costs. Measured against a tree of one image, two VMs and three forks: 577638400 bytes reported against 578457600 actually freed by deleting it, an error of 0.14% where the old figure was 2.64x over. The scan takes about 30 ms. The sweep runs over the whole tree even when the caller passes one record. Exclusivity is not a property a volume has on its own, so a scan narrowed to the requested records would call every volume fully exclusive. All four columns now carry real numbers on macOS, and exclusive <= referenced holds by construction rather than by arithmetic, since the exclusive bytes are a subset of the file's own extents. The sweep is pure interval logic and is unit tested without APFS underneath. fork_shares_blocks_with_origin stops skipping on macOS, where it had been returning early on a null referenced, leaving the one assertion aimed at sharing unrun on the backend whose sharing was broken.
`default-members = ["."]` means a bare `cargo test` covers only the root package. CI ran 30 of 321 unit tests, so every test in ember-core and in the platform backend was green by omission, including the thin_ls and `zfs list` parsing tests this branch added and the spec lists as its no-root coverage. `--workspace` is not the fix. It selects the other platform's backend too, and that backend does not build here: ember-macos needs `clonefile` on Linux, and on macOS ember-linux's image tests want `mkfs.ext4`. Naming the packages per platform is what actually works, matching how `build` already branches on UNAME. `check` and `clippy` get the same selection, so test-module code is compiled and linted too. That immediately caught an `items_after_test_module` failure in this branch's own accounting tests, now moved to the end of the file. `build` and `release` stay as they are: they build the root package, which pulls the platform crate and ember-core in as dependencies, so naming them would compile nothing new. CI calls `make test` rather than `cargo test`, so the lint, check, and test entry points stay consistent.
aljoscha
force-pushed
the
storage-usage-accounting
branch
from
August 18, 2026 10:03
1173a48 to
6322dd8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ember reported provisioned sizes everywhere and actual sizes nowhere.
ember vm listprinteddisk_size_gib,ember infoprinted no capacity at all, and the one command that tried,ember debug storage-efficiency, walked.imgpaths that only exist on macOS and reported zero on Linux.This adds a
usage()method toStorageBackendand wires four commands to it. Design doc:docs/STORAGE-USAGE-SPEC.md.The model
Occupancy, not reclaim. It answers where space went, not what a delete gives back, because on ZFS those are genuinely different questions and one field cannot honestly answer both.
Four numbers per volume:
provisioned,exclusive,referenced,logical. The last two are optional, since not every backend can measure them. Shared bytes and the compression ratio are derived rather than stored so they cannot disagree with their inputs. The invariant that keeps the table readable isexclusive <= referenced.usage()takes the state records and returns the whole installation in one value, so a backend that must walk pool-wide metadata does it once instead of once per volume.Backends
zfscalls.exclusiveisusedbydataset, deliberately notused: a zvol fromzfs create -Vcarries a refreservation for its full virtual size, so image volumes report ausedof 8.4 GiB against areferencedof 1.9 GiB.thin_ls, which also covers volumes that are not currently activated. The reservation is a single slot per pool, so it is released by an RAII guard.st_blocks, leavingreferencedandlogicalunknown.CLI
ember storage usageis strict: being unable to measure is an error, since that is the one thing it exists to do.vm list(newUSEDcolumn),vm inspect, andember infotreat usage as best-effort and render-, because one common reason to run them is that storage is broken.ember debug storage-efficiencyis removed, along with the now-emptydebugsubcommand tree.Review
Two adversarial review passes ran over the first commit; the second commit is the triage. The findings worth naming:
exclusivewasusedbydataset + usedbysnapshots, and snapshot-only space is by definition outsidereferenced, so both image rows shipped with occupancy above what they reference. The guard test for that invariant had been fed a fixture with the offending field zeroed.PoolUsagegainedreservedand the ratio now divides by the occupied remainder.usage()calledensure_pool_active, soember vm listwould load a pool table, attach loop devices, and runthin_checkas a side effect of listing VMs.try_usagecould panic on a config naming an unimplemented backend, which is exactly the config you would run those commands to diagnose. It now goes through a fallibletry_create_storage.Four findings were rejected with justification, mostly around partial results and pre-existing behavior.
Testing
cargo fmt,cargo clippy --all-targets, and 138 unit tests are clean, and the report has been run against a live 481 GiB ZFS pool.Integration tests are in
tests/storage_usage.rs, all#[ignore]d to match the rest oftests/. The macOS/APFS path is unverified — theember-macoscrate cannot compile on Linux (pre-existing,nix::libc::clonefileis macOS-only). It type-checks with that one call stubbed, but nothing about APFSst_blockssemantics has been exercised.🤖 Generated with Claude Code