zvolrescue v0.9.10 — The catalogue closed

zvolrescue v0.9.10 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.10-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	static (FreeBSD's own libc linked in); runs on any 15.x system. Built on 15.x, whose system-call ABI it uses; the real-kernel matrix also runs it on FreeBSD 14.5 and on the mfsBSD 14.2 rescue image, but on 14.x building from source stays the supported way: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.10-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.10-amd64-freebsd.

On Debian and Ubuntu, zvolrescue_0.9.10_amd64.deb and zvolrescue_0.9.10_arm64.deb install the same five static binaries with their manual pages: sudo apt install ./zvolrescue_0.9.10_amd64.deb, then man zvolrescue.

SHA256SUMS covers all 15 binaries and the 2 packages, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.10 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.10-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.10

The catalogue closed.
SPEC §9.1's damage catalogue is covered end to end. The golden image is
published from the run that built it, nobody carrying 2.5 GB through a
browser, and the matrix runs against the published image, pinned by
its sums, on every push that asks and every Monday on its own. The
shapes the manifests named and no image could serve are placed: a
member replaced by an older copy of itself, from the round-6 copies the
release carries; a block's copy on the dRAID, found by content rather
than arithmetic; the dnode blocks, indirect blocks, data blocks and
crypto key object below a dataset's object set, from the oracle's
zdb -dddddd capture. The last two classes without a manifest have
theirs — partial rings with a forged txg, a partition re-created with
another size — and the one case this image could not serve runs now:
61 manifests, 61 pass, none n/a, no defects, inputs unchanged. The
forged txg found a gap, and it is fixed: list passes over a verified
TXG with nothing behind it the way dump passes over one that names a
dataset it cannot read, and dump's record says when it did.

Added

• The golden image is published from the run that built it.
  publish-golden.yml takes the number of a realworld.yml run, downloads
  its golden-<tag> artifact where it already is, checks the sums and
  that no run of the matrix fell outside its expected category, and
  publishes the image in zvolrescue-testdata: the members into the
  release <tag> (created as a draft, filled, checked, published — or
  filled if the tag was made from the web first), the oracle into
  oracle/<pool>/, IMAGE.md and the matrix report into
  reports/<tag>/, in one commit. Nobody carries the 2.5 GB artifact
  through a browser. One secret, TESTDATA_TOKEN, a fine-grained token
  for that repository with contents read and write.
• The matrix runs against the published image, pinned. The
  golden-published job of realworld.yml is the CI job SPEC §9.1
  asks for: it fetches the image-v1 release of zvolrescue-testdata,
  checks every file against the sums this repository pins in
  tests/golden/image-v1.SHA256SUMS (and against the oracle's own
  record of the release), decompresses the members and their round-6
  copies, reads every volume intact to the kernel's hash, and runs the
  whole matrix with the oracle and manifests from that repository's
  checkout, every expectation enforced — the held-out manifests too
  with published-held-out (G4). The release files are kept in the
  Actions cache between runs. It also runs every Monday on its own:
  the image never changes, so a verdict that changes is the tool's
  doing. The verdicts, counts, every n/a's reason and every
  out-of-category run's detail come from one script now,
  tests/golden/summarize-report.py, which both golden jobs share.
• The older-self damage pattern. SPEC §9.1's "member replaced by
  an older copy of itself" was documented in the manifests and never
  placed: no image had a member captured at two points of its life.
  image-v1's release carries every member as it was after round 6 of
  the fill, and run-matrix.py now writes that copy over the range
  the manifest names (older-self:image-v1-round6; the copies beside
  the members as <image>-round6, or --older DIR), refusing a source
  of another image and reporting n/a where the copies are not there.
  Five manifests in zvolrescue-testdata use it: one mirror half
  stale, both halves, two raidz2 columns, one dRAID child, one stale
  beside one missing.
• Structure-targeted damage lands on a dRAID top too. Five of the
  six n/a of every matrix run so far were the MOS and object-set
  copies that live on the image's dRAID top-level vdev: the harness
  maps a block pointer to members by arithmetic on mirrors and raidz,
  and dRAID places columns through a permutation it refused to
  reimplement. It finds them by content now — the block's bytes, read
  from a copy on a mirror or put back together from a raidz's data
  columns and checked against the pointer's fletcher4 or sha256, are
  searched for on the dRAID's children; a column is placed only where
  it is found at one row offset, on distinct children, and a parity
  column that equals a data column (one data sector in a row, the rest
  padding) is placed with it. Checked locally against the kernel-built
  image: the five manifests pass, among them every copy of the MOS
  destroyed and the pool read from an older uberblock.
• Damage below the object set: a dnode block, an indirect block, one
  data block of a named volume. The oracle of image-v1 held
  zdb -dddd — objects, not their block pointers — so a manifest could
  aim at an object set and at nothing beneath it. oracle-capture.yml
  runs OpenZFS's own zdb -dddddd offline over the published members
  (checked against the pinned sums and the pool guid first, on a
  userland at least the image's) and adds the capture to the oracle;
  the builder keeps the same capture for every image from now on. The
  harness reads it: <dataset>:dnode-block, <volume>:data and
  <volume>:indirect with block an index or on:<kind> (the first
  block whose first copy lies on a mirror, raidz or dRAID top), copy
  as before, and pieces = "one" for one piece of the first copy — a
  mirror half, one raidz data column, one dRAID column — the loss every
  geometry survives once. Eight manifests in zvolrescue-testdata use
  them: a data block's one piece and every piece, an encrypted volume's
  block, a dnode block and an indirect block by one copy and by every
  copy, and a dnode block beside a missing member.
  <dataset>:crypto-key is located too — the MOS object the dataset's
  DSL directory names as com.datto:crypto_key_obj, the ZAP holding
  its wrapped keys, every block of it (block = "all") — and two
  manifests destroy secret-raw/disk's by one copy and by every copy,
  so every shape the manifests' format names is placed now; a volume's
  property ZAP on image-v1 is embedded in its dnode's block pointer,
  so the dnode-block cases are its. tests/golden/fetch-image.sh is
  the one place the published image is fetched, pinned and
  decompressed; the matrix job and the capture use it.
• The last two classes of SPEC §9.1's catalogue, and no n/a left.
  forged-txg:<bump> on a label:…/rings range copies the ring's
  newest verified uberblock into the slot ZFS would write that txg to,
  with the txg bumped, the root pointer's birth to match and its
  checksum words inverted, and seals the slot for its place: a TXG
  every member's ring carries, that verifies, and that no block is
  behind — what a sync leaves when its uberblocks reached the disks and
  its MOS did not. grow:<bytes> makes the member longer by that much
  of nothing: the partition re-created with another size, the rear
  labels no longer where the length says. labels-forged-txg (every
  ring forged three TXGs ahead, one member's rear rings gone besides)
  and partition-grown use...

zvolrescue v0.9.9 — The golden image release

zvolrescue v0.9.9 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.9-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	static (FreeBSD's own libc linked in); runs on any 15.x system. Built on 15.x, whose system-call ABI it uses; the real-kernel matrix also runs it on FreeBSD 14.5 and on the mfsBSD 14.2 rescue image, but on 14.x building from source stays the supported way: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.9-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.9-amd64-freebsd.

On Debian and Ubuntu, zvolrescue_0.9.9_amd64.deb and zvolrescue_0.9.9_arm64.deb install the same five static binaries with their manual pages: sudo apt install ./zvolrescue_0.9.9_amd64.deb, then man zvolrescue.

SHA256SUMS covers all 15 binaries and the 2 packages, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.9 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.9-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.9

The golden image release.
The damage matrix of SPEC §9.1 runs in CI over an image a real kernel
built — mirror, raidz2 and dRAID top-level vdevs, every checksum and
compression, encryption, hundreds of TXGs — with every expectation
enforced: 41 manifests, 35 pass, 6 not applicable, no defects. That
image's first build found a pool feature the tool did not know,
OpenZFS 2.4's dynamic_gang_header, which it reads now. The
real-kernel matrix spans Ubuntu 26.04 and 24.04, Debian 13 and 12,
FreeBSD 15.1 and 14.5 and the mfsBSD 14.2 rescue image, and runs on a
machine of your own over ssh; dump passes over a TXG at which any of
what it opens is gone, not only the object set.

Added

• The real-kernel matrix runs on FreeBSD 15 and 14. The script
  (tests/realworld/kernel-matrix.sh, renamed from debian-loop.sh)
  now runs on FreeBSD with the base system's OpenZFS and md devices for
  loop devices: B9's GPT from gpart, F7's failing range a gnop that
  refuses every read between two clean ones joined by gconcat, the
  read-once check from truss, A3 as nobody through chroot -u.
  realworld.yml runs it in FreeBSD 15.1 and 14.5 VMs beside Ubuntu
  and Debian 12, on demand. All four pass (REALWORLD-TESTS, Results);
  on FreeBSD 15.1 (OpenZFS 2.4.2) every one of the 35 scenarios
  applies, C12 included: list refuses a kernel-expanded raidz with
  exit 3 naming raidz_expansion.
• Ubuntu 26.04 and Debian 13 on the matrix, beside 24.04 and 12.
  The Linux jobs of realworld.yml ran only on the releases they were
  written on in September; they are matrices now — the ubuntu-26.04
  and ubuntu-24.04 runners' own kernels, and Debian 13 and 12 VMs,
  the cloud image checked against Debian's SHA512SUMS. On Ubuntu
  26.04 (OpenZFS 2.4.1, kernel 7.0) and Debian 13 (zfs-dkms 2.3.9,
  kernel 6.12) all 35 scenarios apply and pass, C12 included — the
  first Linux kernels on the matrix with raidz_expansion. The old
  releases stay: 12's 2.1 is the oldest ZFS on the matrix and found
  both of D2's defects. The workflow takes an only input naming
  which environments to run.
• E3 on the rescue medium itself: mfsBSD. tests/realworld/mfsbsd.sh
  runs on mfsBSD — FreeBSD booted from an ISO into RAM, /bin/sh and
  the base system alone — where its kernel makes a mirror pool on two
  raw disks and fills a volume, and the static zvolrescue built on
  FreeBSD 15, copied in over ssh, scans both disks, lists the volume
  and dumps it to the kernel's hash; the disks are checked unchanged
  and the pool importable afterwards, and zvolreport runs over the
  evidence. The mfsbsd job of realworld.yml boots
  mfsbsd-14.2-RELEASE-amd64.iso, checked against its published
  SHA2-256, under QEMU/KVM for it. The first run passed
  (REALWORLD-TESTS, Results): the 15.1-built static binary runs on the
  14.2 rescue image. The README and the release notes said "mfsBSD 15
  included" of that binary; no mfsBSD 15 exists, and they now say what
  was checked, and that 14.x stays build-from-source (SPEC N-06a).
• Gang headers the size of a sector: com.klarasystems:dynamic_gang_header
  (OpenZFS 2.4) is read. The feature lets a gang header fill the
  smallest allocation its top-level vdev makes — 4 KiB on an ashift=12
  mirror or raidz, a data row on a dRAID — instead of one 512-byte
  sector, and it goes active on the first gang write that needed more
  than three children; a fragmented pool on 2.4 (Ubuntu 26.04, FreeBSD
  15.1) carries it. v0.9.8 did not know it and refused such a pool
  whole, every dump exit 3 — which is how the first golden image,
  built with ganging forced, came out: 33 of 41 damage runs "refused".
  The reader sizes the header as zio_gang_tree_assemble does, the
  least of the copies' minimum allocations, parses as many child
  pointers as fit before the checksum tail, and — since a pool keeps the
  512-byte headers it wrote before the feature went active, and the
  golden image had hundreds of those and not one larger — tries the
  old size when the vdev's does not verify, as zio_checksum_error
  does. Fixtures write both kinds; four tests read them on a mirror and
  a degraded raidz2 and see the same bytes refused cleanly when the
  labels do not claim the feature. The feature joins the implemented
  list. kernel-matrix.sh C13 fills a mirror with ganging forced on a
  2.4 kernel, sees the feature go active and the dump match.
• The golden image is built in CI, and the whole damage matrix runs
  over it (G1–G3). The golden job of realworld.yml builds the
  image of SPEC §9.1 with tests/golden/build-image.sh on the
  ubuntu-26.04 runner's OpenZFS 2.4.1 — one pool with mirror, raidz2
  and dRAID1 top-level vdevs, 12 volumes over every checksum and
  compression, a clone, raw-key and passphrase encryption, snapshots, a
  rename, a destroy, 5015 TXGs — reads every volume of it intact to
  the kernel's hash, then runs the 41 manifests of
  zvolrescue-testdata over it with every expectation enforced: a run
  outside its expected category fails the job, where the ztest images
  could only be counted. 35 pass, 6 n/a (structure-targeted damage is
  not placed on a dRAID top, whose permutation the harness does not
  map; one case asks for a third leaf of a two-way mirror), 0 defects,
  inputs unchanged (REALWORLD-TESTS, Results). The members (zstd, with
  SHA256SUMS), the oracle, IMAGE.md and the report are uploaded as
  the artifact golden-image-v1, for publication as image-v1 in
  zvolrescue-testdata. Its first build is how dynamic_gang_header
  (above) was found.
• The matrix on a machine of your own, over ssh.
  realworld-ssh.yml runs kernel-matrix.sh — or mfsbsd.sh, on a
  rescue system — on a machine the runners cannot stand in for: a
  distribution with no image here, an OpenZFS built from git, a box
  with disks. It builds the static binaries for what the machine turns
  out to be (Linux x86_64 or aarch64, FreeBSD amd64), copies them and
  the script over, has the script run there as root and brings the
  record back, as the VM jobs do. The machine is described by one
  GitHub Environment's secrets alone — host, user, a key or a password,
  the pinned host key, and an ssh_config for a bastion (ProxyJump
  through a server or an OpenWrt router), with a Tailscale tailnet or
  a self-hosted runner as the way into a network nothing else reaches.
  On the machine the key runs tests/realworld/ssh-gate.sh as a forced
  command: five requests (probe, put, run, record, clean) and no shell,
  root for the run through one sudoers line that allows the gate
  alone. REALWORLD-TESTS, "Running the matrix on your own machine over
  ssh".

Fixed

• dump passes over a TXG at which a destroyed volume's dnodes are
  gone, not only its object set. v0.9.8 made the search pass over a
  TXG whose object set block no longer reads, but checked that block
  alone. The blocks a destroy frees are reused in no particular order,
  and on Debian 12's OpenZFS 2.1.11 and FreeBSD 15's 2.4...

zvolrescue v0.9.8 — The packaged release

zvolrescue v0.9.8 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.8-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	static (FreeBSD's own libc linked in); runs on any 15.x system and on mfsBSD 15. Built on 15.x, whose system-call ABI it uses, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.8-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.8-amd64-freebsd.

On Debian and Ubuntu, zvolrescue_0.9.8_amd64.deb and zvolrescue_0.9.8_arm64.deb install the same five static binaries with their manual pages: sudo apt install ./zvolrescue_0.9.8_amd64.deb, then man zvolrescue.

SHA256SUMS covers all 15 binaries and the 2 packages, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.8 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.8-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.8

The packaged release.
The tool has now been run on real kernels: the matrix's script passes
every applicable scenario on Ubuntu 24.04's in-tree OpenZFS and on
Debian 12's zfs-dkms, on pools the kernel made, every oracle from
OpenZFS — and those runs found what fixtures had not: a member
detached while it was out of the machine listed beside the survivor,
and dump stopping at a TXG whose object set a later write had
reused instead of taking an older one. Both are fixed below. The
FreeBSD release binary is static now, like the Linux ones, and the
one built on 15 runs on 14. And the packaging 1.0.0 was waiting for
is here: manual pages for every program, a FreeBSD port, a .deb and
an AUR PKGBUILD, each built and installed in CI on every push.

Added

• Manual pages, and packages that install them: a FreeBSD port, a
  .deb, an AUR PKGBUILD (REALWORLD-TESTS E4). man/zvolrescue.1,
  zvoltimeline.1, zvolreport.1, zvolcarve.1 and zvolfiles.1 in
  mdoc, each kept true to its program by tests/man-check.sh, which
  reads --help of every subcommand and refuses a page missing an
  option, carrying another version, or failing mandoc. Under
  packaging/: deb/build.sh makes zvolrescue_<version>_<arch>.deb
  of the five static binaries and the pages with dpkg-deb alone, and
  every release now ships one for amd64 and arm64, covered by
  SHA256SUMS; freebsd/sysutils/zvolrescue is a port (USES=cargo,
  its crate list generated from Cargo.lock, its distinfo written by
  a script that hashes the tarball and every crate the way make makesum does); aur/ holds the PKGBUILD and .SRCINFO, with a
  script that points them at a new tag. CI builds and installs all
  three on every push — the port on FreeBSD 15 through the ports
  framework, the .deb with dpkg, the PKGBUILD with makepkg in an
  Arch container — and each install runs every program and finds every
  page. Submission to the ports tree and to the AUR is still to do.

Fixed

• A member detached while it was out of the machine is set aside,
  not listed beside the survivor. zpool detach erases the labels
  of the device it detaches — but only of a device that is there. A
  disk pulled from a mirror before the detach keeps its labels: the
  same top-level guid (a detach hands the mirror's guid to the
  survivor), an older txg, a leaf the survivor's configuration no
  longer names. The assembler treated the two labels as the same
  top-level vdev, took the newer tree, and then listed the old disk
  under devices as though it belonged, with nothing in stale to
  say otherwise. Found on a kernel-made pool in the first real-kernel
  run of the matrix (REALWORLD-TESTS B8, Ubuntu 24.04 and Debian 12);
  reproduced with a fixture, and such a label is now stale with the
  reason: it names a leaf the top-level vdev's newest configuration no
  longer has — detached or replaced.
• dump passes over a TXG that names the dataset but has lost its
  object set, and takes an older one. After a zfs destroy the pool
  goes on writing, and the freed object set block is among the first
  to be reused; the uberblock ring may then name the dataset at a TXG
  whose object set is gone while an older one still leads to it whole.
  dump stopped at the first TXG that named the dataset and reported
  every copy failed its checksum (exit 3) without looking further.
  Found in the Debian 12 run of the matrix (REALWORLD-TESTS D2, OpenZFS
  2.1.11); reproduced with a fixture. Now such a TXG is said on stderr,
  listed under unreadable_at in the record, and the search goes on to
  the next older verified TXG; readable at none, the run says at which
  TXG the dataset was named and why it does not read there. --txg
  still means that TXG and no other.
• The FreeBSD release binary is static, as the README always said
  it was. Up to v0.9.7 the amd64-freebsd files were linked against
  FreeBSD 15's base libc.so.7, libthr.so.3 and libgcc_s.so.1 —
  fine on any 15.x system and on mfsBSD 15, which carry them, but not
  the "static binaries with no runtime dependencies" the README
  promised, and file said so. The release workflow now builds the
  FreeBSD binaries with crt-static, and both it and the publish step
  refuse a file that file does not call statically linked, on every
  platform. The FreeBSD 15 CI job builds the same way on every push
  and runs that binary on its real devices, so a crate that will not
  link statically is caught there, not at release time. A static
  build still carries the system-call ABI of the branch it was built
  on: on 14.x, build from source as before. For a commit between
  releases, or for a tag whose release still carries the dynamic file,
  .github/workflows/freebsd-static.yml builds the five programs the
  same way on demand and uploads them with their SHA256SUMS.

Documented

• The first real-kernel runs of the matrix, and what they taught the
  script. .github/workflows/realworld.yml runs
  tests/realworld/debian-loop.sh on demand in two environments:
  the Ubuntu runner's in-tree OpenZFS, and a Debian 12 cloud image
  under QEMU/KVM with zfs-dkms built against Debian's kernel. The
  first runs found the script's expectations wrong in four places and
  the tool's in one (above): zpool create on a bare loop device
  labels it but the partition node it then waits for never appears on
  either kernel, so B9 writes the GPT itself; Debian 12 carries
  OpenZFS 2.1.11, which has no blake3, so a property value this ZFS
  lacks is noted rather than fatal; without a key dump refuses an
  encrypted volume up front (exit 1, naming the suite) instead of
  writing zeros, which is the better behaviour and what C7 now says;
  and zfs get -s local reports volsize as local though the kernel
  keeps it in the volume's own object, not the property ZAP, which
  C11 now allows for alongside volblocksize. B4's corruption is
  aimed at the volume's own blocks through their DVA, since 1 MiB at a
  round offset landed in free space on OpenZFS 2.1. D1 accepts a
  partial image when the export's own TXGs reused a block of the
  volume destroyed just before it — compared byte for byte against a
  copy taken before the destroy — and D2 the clean exit 3 the dump
  fix above gives. With that, both environments pass every applicable
  scenario (34 of the matrix; C12 needs OpenZFS 2.3), and the two rows
  are in the Results table of REALWORLD-TESTS. The row the script
  prints now names where the module came from (zfs-dkms or the
  in-tree module) and the module's version where it differs from
  userland's.
• The FreeBSD static binary built on 15 runs on 14 (REALWORLD-TESTS
  E3). A CI job takes the crt-static binary the FreeBSD 15 job
  built, with the mirror fixture and the hash of the volume it read,
  and runs version, scan and dump on FreeBSD 14.5-RELEASE: all pass,
  the hash matches. The matrix row says so, with the caveat that this
  is a full 14 VM and not mfsBSD itself.

Full history: CHANGELOG.md.

zvolrescue v0.9.7 — The interruptible release

zvolrescue v0.9.7 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.7-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.7-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.7-amd64-freebsd.

SHA256SUMS covers all 15 of them, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.7 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.7-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.7

The interruptible release.
A run can now be stopped and taken up where it stopped, in every tool
that runs long: SPEC §7 had promised exit 6 for an interrupted run
since 0.1 and nothing produced it — a SIGINT killed the run, and
--resume had whatever the checkpoint five seconds earlier had said.
zvolrescue dump, zvolcarve dump and zvolcarve scan end at the
next block or chunk with their state written; zvolfiles extract at
the next entry with the manifest saying what was written; zvolcarve dump gains the --resume its spec had promised, on the very state
file zvolrescue dump writes. And the first real-kernel environment
of the matrix is scripted: Debian with zfs-dkms and loop devices,
every oracle from OpenZFS alone, ready to run on a VM — which is the
step 1.0.0 waits on.

Added

• The real-world matrix on Debian with zfs-dkms and loop devices,
  scripted (tests/realworld/debian-loop.sh). The cheapest
  environment with a real kernel ZFS, and the first one REALWORLD-TESTS
  should see run: pools the kernel makes on loop devices, which the
  tool treats as the block devices they are (N-10). The script builds
  every layout of B, every property of C, every recovery of D, the
  dm-error member of F7 and the read-only checks of A, takes every
  oracle from OpenZFS alone (sha256sum of the zvol before export,
  zdb -d, zdb -lu, zfs get -s local), and prints the row for the
  Results table. raidz_expansion (C12) needs OpenZFS 2.3 and is
  skipped on 2.2; dRAID is skipped where zpool builds none. Found
  while writing it: dump had no SIGINT handler (the next entry). The
  matrix has a section describing what every scenario becomes on loop
  devices and what a loop device cannot give (E1/E2, F1–F6, F8–F10
  stay for a machine with drives).
• A SIGINT ends dump where --resume can take it up (F-32). The
  signal used to kill the run: exit 130, the image cut wherever the
  kernel happened to be, and --resume starting from whatever the
  checkpoint five seconds earlier had said — or from nothing, when the
  run was younger than that. Now SIGINT and SIGTERM set a flag the
  extraction reads before every block: the run ends at that block
  boundary, writes its state naming the block it was about to read,
  reports interrupted_at in the output and the evidence record, and
  exits 6 as SPEC §7 has said since 0.1; a bulk -r run stops there
  instead of starting the next volume. A second signal is the default
  action again, for a run that does not come round to the flag. The
  five-second checkpoint stays, for a kill no handler sees. The flag
  lives in zvol-common on the signal-hook crate (no unsafe of
  ours; the workspace still forbids it); extract_from takes it as an
  optional AtomicBool. Unit-tested in zfs-read, and CI sends a
  real SIGINT to a 512 MiB dump once 128 MiB are on disk, then resumes
  it to the same hash.
• The companions stop the same way. zvolcarve dump gains the
  --resume COMPANIONS §3.2 had promised, on the very state file
  zvolrescue dump writes — the mechanics moved into
  zvol_common::resume, so there is one extraction pipeline and not
  two that agree by luck — and a SIGINT ends it at the next block with
  the state written, exit 6. zvolcarve scan stops at the next chunk:
  the state file says where each member got to, members not reached
  keep what an earlier run reached, and --resume goes on from there
  (C-08 covered the candidate cap; now it covers the operator too).
  zvolfiles extract stops at the next entry, or the next block of
  the file being written: that file is recorded as failed with how
  much of it was written, the manifest says interrupted, exit 6, and
  --path asks for the rest (Z-11). CI interrupts a real carve scan
  and a real carve dump and resumes both.

Documented

• The open questions of §12 are closed (D-9…D-11). What the
  repository already did is now a decision with its reasons: Linux CI
  from phase 0 with FreeBSD beside it and both shipped (D-9);
  raidz_expansion deferred past 1.0.0, a pool with it active refused
  with the reason, because a reflowed layout returns wrong bytes with
  checksums that do not object and no such pool has been measured
  (D-10); the library crates internal and unpublished until 1.0.0,
  every crate marked publish = false so that no stray cargo publish can decide it first (D-11). With D-8 that leaves §12 with
  nothing open, which was one of the three conditions §10 sets for
  1.0.0.

Fixed

• The fixtures' dnode and objset blocks carry the object counts ZFS
  keeps in fill. v0.9.6 made the dense volume's indirect pointers
  count the blocks beneath them; the level-0 blocks of dnode arrays
  and every objset's root pointer still said fill = 1. ZFS writes
  the number of allocated dnodes in a dnode block, a multi-slot dnode
  counted once, and the sum of those under an objset's meta dnode —
  which is what zdb -d prints as the object count. The fixture now
  writes both, through two allocator calls that count what they
  store, and a test reads the sample pool's MOS and the volume's
  objset back and compares each pointer's fill with the dnodes it
  finds in the block. This tool never reads fill; the cross-check
  with zdb does.
• A device that refuses a read while the members are being bound
  stops the run there too (SPEC F-33, N-10). After every member's
  labels are read, open_members still reads through the pool — a
  removed vdev's mapping, a layout searched for, a leaf bound by
  reading — past its own check for a stop. A refusal there was
  recorded and swallowed, and the run went on into the command with a
  stopped ledger, to end wherever it ended. Now a stop recorded by
  those reads ends the open the way one during the labels does, and
  the command's end of run reports it. zvolcarve dump refusing to
  write over evidence goes through that end of run as well, like its
  other early returns.

Full history: CHANGELOG.md.

zvolrescue v0.9.6 — The review release

zvolrescue v0.9.6 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.6-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.6-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.6-amd64-freebsd.

SHA256SUMS covers all 15 of them, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.6 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.6-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.6

The review release.
Everything since v0.9.0 read again, with tests for what the reading
found. Two defects, neither of which had shipped a wrong byte but both
of which broke the medium policy's promise: a device that refused a
read could still be reached by any reader that went around the pool
reader, and a refusal on the way to the dataset came back as exit 3
with no incident on record when the run was a single volume. Both are
closed, in the io layer and at the end of every run, and pinned on a
real block device in CI. The fixtures stripe gang blocks and remapped
blocks onto raidz now, the dense fixture's indirect pointers count
what is beneath them, and the fuzzer starts from the inputs that once
crashed it.

Changed

• A device that refused a read is closed for the rest of the run, in
  the io layer (SPEC N-10). The medium policy stopped the run at the
  first refusal, and the pool reader refused every later read — but
  only the pool reader. The io layer recorded the incident and went on
  answering reads of the same device, so anything that read a member
  without going through the pool reader (the label reads of a scan,
  the zero-point search, carving) could still reach a disk that had
  just refused, and a unit test pinned that a second read after the
  stop goes through. Now the ledger knows which path the stopping
  incident was on, and every source that is a device answers every
  later read of that path with that incident before touching it —
  memory hits included, so nothing masks a caller still trying. Other
  devices and every image go on being read: the run is over, but the
  report still wants their labels. --device-may-fail is unchanged in
  spirit: a skipped refusal closes nothing, the flag exists so that the
  device's other addresses are still read, and the incident that stops
  the run closes it like any stop. Four tests, each failing without
  the gate: the second read after a stop is refused and the device not
  touched; the same path through another handle is closed while
  another device and an image on the same ledger read on; with
  --device-may-fail the device stays open until the limit and is
  closed by it; and a real FileSource that is a device refuses
  before the read it would otherwise answer.

Fixed

• Every run ends the same way, whichever way it failed (SPEC F-33,
  N-10). The fix below closed the early returns of dump and list;
  a review of the line found the same door open elsewhere. zvolfiles list, objects and extract returned a bare 3 when the MOS or the
  dataset walk failed, with no incident on stderr and no record. And
  when a device refused a label read, so that the members never
  opened, every tool printed the incident and exited 7 but wrote no
  evidence record at all. Now the ledger is made once per run, on the
  pool spec, before anything is opened, and one end_run closes every
  run — the report of the incidents, the stop as the exit code, the
  record with them — including the runs that failed while opening,
  choosing a pool or a transaction group, or before writing a byte:
  dump, list, zvoltimeline, the three zvolfiles commands and
  the four zvolcarve commands all go through it. A run that produced
  nothing records that it produced nothing. Checked: in cargo test,
  a dump of a dataset that does not exist and a list at a
  transaction group nobody has both end with their record; in CI on a
  real block device, zvolfiles list and zvoltimeline on a member
  whose label 0 refuses exit 7 with one incident on stderr and the
  record saying so, and a scan of that member reports it refused,
  with the incident, and opens no further member after the stop.
• A device's refusal on the way to the dataset is exit 7 and an
  incident on record, on a single volume too (SPEC F-33, N-10). The
  medium policy of v0.9.1 reported a refused read after the run: every
  incident on stderr, the stop as the exit code, both in the evidence
  record. A single-volume dump and a list never got there when the
  refusal landed on a metadata read — the MOS objset, the dataset tree,
  the volume's dnode: "cannot read the pool" returned early with exit
  3, no MEDIUM INCIDENT line, and no evidence record at all, while
  the ledger held the incident and nobody read it. A bulk -r run
  reached the report, so the one-volume case was strictly worse. Now
  every early failure ends the way every run ends. Checked in CI on a
  real block device: the eight sectors of the MOS objset mapped to
  device-mapper's error target, dump and list both exit 7 with the
  incident on stderr and in the record — the block's address taken
  from the reader's own trace, so the test follows the fixture.
• The dense fixture's indirect pointers count the blocks beneath
  them. Every pointer the fixture wrote carried fill = 1, the
  indirect ones included; ZFS keeps the number of non-hole level-0
  blocks under an indirect pointer there. This tool never reads fill,
  so nothing it does changed, but zdb reads it and a fixture that
  lies to the cross-check is worse than none. A test builds 2048
  blocks under two levels and checks every pointer at every level.

Added

• The direct readers, tested against a closed device. The io-layer
  gate (above) was shown with strace in CI; now a unit test builds a
  member whose first sector refuses, runs the label scan on it as a
  device, and asks what a scan asks next — the partition table, the
  GEOM sector, the anchor search with leave — and finds every one
  refused with the incident and the device read exactly once. And a
  gang header's asize is asserted on a mirror and on raidz2, where
  nothing in the read path would notice a wrong one.
• The four inputs the fuzzer crashed on, committed as seeds.
  v0.9.0's fuzz commit fixed four panics and kept the crashing inputs
  only under fuzz/artifacts, which is ignored; CI had to rediscover
  them inside its thirty seconds. They are under fuzz/seeds now,
  named for what they broke — the indirect mapping's address
  arithmetic, a GPT entry's LBA arithmetic twice, a microzap shorter
  than its header — beside the deterministic unit tests that already
  guard each fix.
• A healthy device asked for its zero point is refused the search,
  with exit 5 — pinned (SPEC N-10). The labels verify, the pool is
  reported, and --zero-point still gets zero point: not searched,
  surface_scan_refused in the JSON and exit 5: the operator asked
  for a surface search and did not get one, and a script must be told.
  That was the behaviour; the only test covered a device whose labels
  do not verify. Now the CI device job runs it on a real block device
  with verified labels and shows through strace that the refusal
  came before any read past the four labels.
• Smaller guards from the same review. A read that only partly
  overlaps a remembered window is never served from it, asserted
  rather than assumed; a symlink to a device — /dev/mapper/NAME,
  /dev/disk/by-id/…, the names an operator types — is refused a
  surface scan like the device itself, by scan and by zvolcarve;
  and scan --retries is refused like dump --retries.
• Gang blocks and remapped blocks on a degraded raidz, tested. The
  defect v0.9.5 caught before its tag lived on the unverified read
  path — the one a gang header, a block remapped by a vdev removal and
  read_dva g...

zvolrescue v0.9.5 — The measured release

zvolrescue v0.9.5 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.5-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.5-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.5-amd64-freebsd.

SHA256SUMS covers all 15 of them, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.5 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.5-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.5

The measured release.
The 512 MiB memory bound and the throughput target had been asserted
and never measured, and zstd was decoded by a pure-Rust crate on the
strength of an assumption. Now every push builds dense 512 MiB volumes,
extracts them and prints the rate and the peak resident set; runs the
reader's own zstd decoder against libzstd on the same blocks; and dumps
one volume twice to check the bytes and the documents come back
identical. The first measurement found its own defect: raidz2 read an
uncompressed volume at a third of the mirror's rate, because every
stripe fetched its parity before anything said it was needed. A healthy
stripe now reads its data columns alone — and a review of the whole
range before the tag caught what that change had broken on the other
read path, a degraded raidz behind a gang or remapped pointer, which is
fixed here and never shipped. Deduplicated volumes are in the
fixtures.

Changed

• A healthy raidz stripe is read from its data columns alone. The
  first measurements (below) showed an uncompressed volume on raidz2
  extracted at a third of the mirror's rate. Two causes, both in the
  striped read: every row fetched its parity columns before anyone
  knew whether they were needed, and the stripe was assembled one byte
  at a time through an iterator. Now the data columns are read first;
  the parity is fetched only for a row that lost a column or, after
  the checksum fails, for the combinatorial reconstruction — as
  OpenZFS reads a healthy stripe, and as a device asks (SPEC N-10):
  half the reads. Both read paths do it — the verified one and the
  unverified one behind read_dva, gang headers and remapped blocks —
  and each fetches the parity of a row that lost a column before it
  reconstructs. The assembly copies whole columns. Measured on the same
  volume: about 290 MB/s on raidz2 against about 310 on the mirror,
  from 130; on a quiet machine raidz2 now extracts at the mirror's
  rate, uncompressed, lz4 and gzip alike. Two tests: one counts the
  reads of an extract on a four-wide raidz2 and finds none of the
  parity columns among them; the other reads a pointer through the
  unverified path with each one of the four members missing in turn,
  and checks the reconstructed bytes against the pointer's checksum.

Added

• The zstd decoder, measured against libzstd (SPEC §12 Q3, D-8).
  tests/zstd-bench.sh cuts a fixture volume's bytes into 128 KiB
  blocks, compresses them with the zstd command, gives each the
  framing OpenZFS gives a zstd block and decodes the lot through the
  reader's own decoder (zstd-bench, an example of zfs-ondisk),
  every block checked against its original; libzstd's built-in
  benchmark runs on the same bytes. It runs on every push and prints
  one table. Measured: about 400 MB/s for ruzstd on one thread
  against about 1 GB/s for libzstd on the fixture text at level 3 —
  more than the media this tool reads deliver, so ruzstd stays and
  the question is closed as D-8.
• A deduplicated volume in the fixtures (SPEC F-28). mkfixture … dedup builds the sample pool with tank/vm/disk0 deduplicated: its
  data pointers carry the dedup bit and sha256 checksums, and
  dedup=sha256,verify is set among its properties. There is no
  deduplication table on the pool and none is looked for — a dedup
  pointer is a pointer — so list --properties names the property
  and dump gives the plain volume's image, byte for byte. Both are
  tests now, in the library and on the command line.
• Memory and throughput, measured on every push (SPEC N-03, N-08,
  N-05). Until now the 512 MiB bound and the throughput target were
  asserted and never measured. mkfixture bench and bench-raidz2
  build a dense volume — every block present under a real indirect
  tree, 128 KiB blocks, of any size, with compression off, lz4 or gzip
  (the fixture gained a compressing put) — and print the SHA-256 the
  extracted image must have. tests/bench.sh extracts six such volumes
  of 512 MiB and reports, per run, the extraction rate, the rate of a
  plain copy of as many bytes through the same page cache, their ratio,
  and the peak resident set, which dump now reports itself
  (peak_rss_kib, from /proc/self/status on Linux). It fails on a
  wrong hash, on a resident set over 512 MiB, or on a rate under a
  floor; the N-08 ratios are to a disk's read rate and wait for the
  disks of REALWORLD-TESTS E1 and E2, which now say so. Measured here:
  peak resident set about 70 MiB on every volume, and the reader alone
  — output to tmpfs, so the disk is out of it — about 460 MB/s on one
  thread, mirror or single member alike. Reproducibility (N-05) is
  checked the same way: the same volume dumped, scanned and listed
  twice gives identical images and identical documents once seconds
  and peak_rss_kib, the two fields meant to vary, are set aside.

Full history: CHANGELOG.md.

zvolrescue v0.9.1 — This tool reads healthy media

zvolrescue v0.9.1 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.1-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.1-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.1-amd64-freebsd.

SHA256SUMS covers all 15 of them, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.1 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.1-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.1

This tool reads healthy media.
A data-recovery practitioner read v0.9.0 and saw the danger in it: a
sector a live disk refused was read again in pieces, then by sector,
then again with --retries, and a surface scan read a device end to
end. That is what a failing disk survives least. Physical and logical
recovery are separate trades: a disk with defects is imaged first, by a
tool that knows the kind of defect and when to stop, and this tool
works on the image and its map. So now: on a device the first refused
read stops the run with exit 7 and the incident, nothing is read twice
anywhere, surface scans refuse a block device, and --retries is gone.
Checked on real block devices in CI — device-mapper's error target on
Linux, gnop on FreeBSD — with strace proving no address read twice.

Changed

• This tool reads healthy media (SPEC N-10, F-33 rewritten). A
  data-recovery practitioner's reading of v0.9.0: re-reading a sector a
  live disk refused — in pieces, then sector by sector, then again with
  --retries — is what a failing disk survives least, and a surface
  scan of one is worse. Physical and logical recovery are separate
  trades: a disk with defects is imaged first, by a tool that knows the
  kind of defect and when to stop, and the metadata is parsed from the
  image. So: on a device, the first read it refuses stops the run
  with a new exit code 7. The incident — device, byte offset and LBA,
  length, errno, time, and which volume and block — is on stderr, in
  the JSON (stopped_by_medium) and in the evidence record
  (incidents), and the message says what to do next: image the disk
  with a map and continue on the image with --map and --resume.
  Nothing is read twice: no retry, no pieces, no sectors, and not the
  other side of a mirror either — the operator is told first and
  decides. --retries is gone. --device-may-fail is the one way past
  the first incident: the refused range is skipped once — the mirror
  heals it if it can, else zeros with the incident as the reason — and
  the run stops anyway after eight. On an image with its map (F-72)
  nothing changes: the map refuses ranges before they are read, and a
  bad sector still costs only its sector. Checked in CI on a real block
  device: a mirror member on a loop device with eight sectors mapped to
  device-mapper's error target stops the run with exit 7 and the LBA;
  with --device-may-fail the mirror heals the block and strace shows
  no address on the device read twice; on FreeBSD, gnop -e 5 makes
  the first label read the incident.

• No address on a device is read twice, anywhere (SPEC N-10). The
  metadata walk used to come back to the same MOS blocks over and over
  — one block nineteen times in a small dump — and a scan read each
  label twice, then its first and last sectors again for the partition
  table and the GEOM check. The reader now keeps the metadata blocks it
  has read (bounded: small blocks only, dropped whole past 64 MiB), each
  label is read once, and a source answers any read that lies inside a
  recent read from memory. A scan of a device is four reads, one per
  label; a dump of a two-block volume off a mirror member is sixteen,
  none overlapping.

• Surface scans refuse a block device (SPEC N-10). The search for
  uberblock anchors when a member's labels are gone (F-61), the root
  search when no uberblock survives (F-64), pointer self-consistency
  (F-63) and carving (zvolcarve scan) answer a block device with exit
  5 and the reason, unless --surface-scan-on-device is given. scan
  reports it as zero point: not searched (JSON surface_scan_refused)
  and exits 5 too. Image the disk, and search the image.

Full history: CHANGELOG.md.

zvolrescue v0.9.0 — Checked, fuzzed, interpreted, signed

zvolrescue v0.9.0 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.9.0-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.9.0-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.9.0-amd64-freebsd.

SHA256SUMS covers all 15 of them, and SHA256SUMS.sigstore.json is its signature — made by this release workflow at this tag, keylessly, through Sigstore:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS
cosign verify-blob --bundle SHA256SUMS.sigstore.json \
  --certificate-identity https://github.com/click0/zvolrescue/.github/workflows/release.yml@refs/tags/v0.9.0 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS
gh attestation verify zvolrescue-0.9.0-x86_64-linux-musl --repo click0/zvolrescue   # provenance of any one binary

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.9.0

Checked, fuzzed, interpreted, signed.
The audit release, the first of the three tags between v0.8.5 and 1.0.0
(SPEC §10). Nothing the tool reads or writes changes; what changes is
what is known about it on every push. The dependency graph is held to a
licence list and a ban list, and every binary's import table is read
and found to ask for no socket, no exec and no dlopen. Every parser is
fuzzed, and the four panics the fuzzer found on its first run are fixed.
The unit tests of the on-disk parsers run under Miri. And the release
files are signed, keylessly, by the workflow that built them.

Added

• What the dependency graph carries, and what the binaries ask their
  platform for, checked on every push (SPEC §8.3 (3), N-07). The tool
  reads evidence and writes images; it has no business on a network and
  never runs another program. The source said so, and nothing checked
  it. Now deny.toml holds the rule for the graph: every licence in the
  tree is a permissive one on the list — the CDDL of the Edon-R port is
  allowed on the edonr crate alone, where OpenZFS's files are kept
  apart — no advisory stands against a crate, and the crates that exist
  to open a socket, run a program or load code are named and banned, so
  a dependency that grows one is stopped on the branch it arrives on.
  And every binary the workspace builds — zvolrescue and the four
  companions — has a test that reads its ELF symbol tables and fails on
  any of socket, connect, execve, fork, posix_spawn, dlopen
  and their kin, whether imported from the C library on a dynamic build
  or linked in on a static one. The reader is checked against a program
  that does connect and does run a program, so a clean result is known
  to mean something. A source grep for process::Command and
  std::net runs beside it. Today's binaries import none of it.

• The parsers are fuzzed on every push (SPEC §9). Everything in
  zfs-ondisk that reads bytes off a disk is now a cargo fuzz target
  — a packed nvlist and the label configuration read out of it, a block
  pointer and a gang header, a dnode and an objset header, the three ZAP
  forms, an uberblock slot and a vdev_phys, every bonus buffer (space
  map, indirect mapping, DSL directory and dataset, bpobj, deadlist,
  znode, system-attribute header), every decompressor, and a GPT, an
  MBR and a GEOM sector — in both byte orders where the structure has
  one. Each starts from a well-formed seed built with the crate's own
  encoders and read back through the parser before it is committed
  (fuzz/seeds, from the example fuzz-seeds), and CI runs every
  target for a bounded time under the address sanitizer on a nightly
  toolchain, failing on a panic, an abort or an input that runs past
  the time limit and keeping that input as an artifact. The workspace
  itself stays on stable; the fuzz crate is outside it.

  The first run found four ways to make a parser panic, all on input no
  pool writes and every one of them a crash on a damaged disk: an
  indirect-mapping entry whose destination offset sat at the top of the
  address space overflowed the addition in remap, and the sum of
  entry sizes could overflow too; a GPT slot whose LBAs did not fit the
  byte address space overflowed the multiplication, and one spanning
  the whole space overflowed its length by one; and a block whose first
  eight bytes said "microzap" and which ended before its header chunk
  did was sliced past its end. Each now answers as damage — unmapped,
  a skipped slot, a truncated block — with a unit test beside it.

• Release files are signed (SPEC N-07). Nobody holds a signing key:
  the release workflow signs SHA256SUMS — which covers every binary —
  with a short-lived certificate that Sigstore's Fulcio issues to that
  workflow run against GitHub's OIDC token, and ships the signature,
  the certificate and the transparency-log entry as one bundle beside
  it, SHA256SUMS.sigstore.json. The certificate names this
  repository's release workflow and the tag it ran for, and that is
  what cosign verify-blob checks; the release notes carry the exact
  line. The workflow verifies its own signature before publishing, so
  a release whose signature does not verify is not published. Each
  binary also gets a GitHub build-provenance attestation — the commit,
  the workflow and the runner that built it — for gh attestation verify. Nothing changes for sha256sum -c.

• Miri on the on-disk parsers' unit tests, every push (SPEC §9).
  The workspace forbids unsafe code, so what the interpreter catches is
  in the dependencies the parsers lean on and in the standard library's
  contract as the tests use it: an uninitialised read, an aliasing
  violation, an out-of-bounds access the native build happened to
  survive. The 125 tests of zfs-ondisk that run there pass, in five
  and a half minutes on a laptop core. A handful that
  loop over megabytes of pure arithmetic — noise fed to the carver and
  the filesystem signatures, the space-map replay, every RAIDZ loss
  pattern, the dRAID permutation maps, a 1 MiB fletcher — run a tenth of
  their volume there, and the vdev_phys round trip, three SHA-256
  passes over 112 KiB that took six minutes interpreted, is skipped
  there; each says so beside its loop, and every one runs in full on
  the native build.

Full history: CHANGELOG.md.

zvolrescue v0.8.5 — What the disk never gave, and what the disk still says

zvolrescue v0.8.5 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.8.5-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.8.5-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.8.5-amd64-freebsd.

SHA256SUMS covers all 15 of them:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.8.5

What the disk never gave, and what the disk still says.
An image's ddrescue map is honoured: sectors the imager could not read
are refused by name, not read as data, and a bad sector on any member
costs its sector rather than its block. A disk's own names — a GPT
label, a gptid, a glabel — tie a member whose labels are gone to the
leaf its siblings describe, and a member with no anchor at all is found
at the base their asize bounds. Three-way mirrors and pools of several
mirrors are fixtures now, and ashift 9 and 13 are cross-checked against
zdb.

Added

• The imager's map (SPEC F-72), and retries per sector (F-33). An
  image of a failing disk is made with GNU ddrescue, and the mapfile
  beside it is the only record of which bytes the disk actually gave.
  Where it could not read, the image holds zeros that look like data,
  and a block crossing them failed its checksum with nothing to say
  why. --map MEMBER=MAPFILE reads the map — its own format, comments
  and status line and hex positions — and every unfinished range is
  refused before it is read, the way a disk refuses a bad sector, so
  the block takes the F-33 path: kept as far as the imager read it,
  zeros for the rest, and the reason names the imager. scan reports
  the map per device: bytes never read, by status, and which of the
  four labels the unread ranges touch. The mapfile is hashed into the
  evidence record as an input. And --retries N (default 1) reads a
  refused sector that many more times before giving it up, for the
  marginal sector on a live disk that gives on the third try — an
  image file pays nothing for it. Prompted by a Habr write-up of a
  NAND recovery that read every failing page up to 128 times: the
  lesson for a tool that starts from images is that the image's own
  record of what was never read has to be honoured.

• A member with no anchor at all is found at the base its siblings
  bound (SPEC F-62, the other half). F-62 could bind a label-less
  member to a leaf by reading through it, but only at a base something
  on the member had confirmed: a surviving uberblock ring, or a
  partition table. Every label gone — rings included — and the member
  somewhere inside a larger image with no table left, and the binding
  tried base 0, read garbage and said nothing reads through it.

  The siblings' labels say how big a leaf of that top is (asize, plus
  the labels at both ends), which bounds where its vdev can begin. The
  trial now tries 0 and every 1 MiB multiple that leaves room for such
  a leaf — the first and last 512 of them on a disk large enough to
  have more, since a member is at the front of its disk or its tail is
  at the end — and the base that walks is the one the member is read
  at. Confirmed by the checksums the walk verifies, never assumed, and
  said out loud: read as leaf … with its vdev at byte N. The fixture
  can now write a real asize for it.

• Fixtures for a three-way mirror and for a pool of two mirrors.
  Both shapes were read before — a mirror is any number of sides, and
  a pool is any number of top-level vdevs — but only ztest ever
  produced them, by chance: a side attached during its run, a vdev
  added. Now mkfixture mirror3 builds the widened mirror and
  mkfixture striped a pool whose MOS is on mirror-0 and whose
  volume's data is on mirror-1, three leaves wide, so that a read
  has to cross tops to finish. Every push checks: any one side of the
  three-way mirror is a whole copy; the damage-matrix shape (one side's
  labels gone, another absent, the third describing the pool) binds the
  bare side by reading (F-62) and the volume comes out; the two-mirror
  pool reads with one side of each; and with every member of mirror-1
  gone the MOS still lists the volume while its blocks are refused by
  the name of the top nothing describes — zeros with the reason, exit 4
  — because a member's label describes only its own top-level vdev,
  which is how ZFS labels work and what makes that top unrecoverable
  without --hints.

• A bad sector costs its sector, not its block (SPEC F-33). A disk
  with one unreadable sector refuses the whole 8 KiB read that touches
  it, and with no other copy dump wrote 8 KiB of zeros — 7.5 KiB of
  which the disk would have given. The real-world matrix had carried
  "read per sector on EIO — to implement" since it was written.

  A read a member refuses is now read again in 4 KiB pieces and, inside
  a piece that still fails, sector by sector; what comes back is kept
  and only the refused sectors are zeros. Each zeroed sector is its own
  range in bad, blocks_salvaged counts the blocks it happened to,
  and the exit code is 4 as for any image with zeros in it. What was
  kept is unverified — the checksum covers the whole block and part of
  it is gone — and the range's reason says so, because a forensic
  report that called those bytes verified would be lying.

  Only what can be kept in part is: an uncompressed, unencrypted block
  on a disk or mirror. A compressed block cannot be decompressed in
  part, ciphertext cannot be decrypted in part, and under raidz or
  dRAID a partial column is reconstructed rather than kept, so those
  are zeroed whole as before, with the reason naming which. A mirror
  still heals from its other side first, and nothing is salvaged that
  could be read whole; a read that fails once and then gives every
  sector counts only if its checksum then agrees. Tested against a
  source that refuses reads touching given ranges the way a disk with
  bad sectors does; the read error a mirror reports also stopped
  saying "no present member" about a member that is present and
  failing.

• The names a disk gives itself tie a bare member to its leaf (SPEC
  F-71). On FreeBSD a member is usually handed to ZFS as
  /dev/gpt/NAME, /dev/gptid/… or /dev/label/NAME, and the pool's
  labels record that string as the member's path. scan already read
  the GPT partition name and did nothing with it; the partition's own
  GUID and glabel's metadata were not read at all. Now every device
  lists the names its disk carries — GPT name and GUID, and any GEOM
  class in the last sector of the device or of a partition — and where
  a pool has a vacant leaf whose recorded path is one of those names,
  the scan says which device that is and spells out the
  --assume-member that binds it. The binding is still confirmed by
  reading, exactly as F-62 does: the name says what the disk was
  called, not what is on it. A name that merely contains a pool's name
  (tank-spare beside tank) is a hint and binds nothing.

  Two more things fall out of reading the GEOM sector. The metadata
  records the size of the provider it sits on, and ZFS was given one
  sector less than that and placed its rear labels against that end —
  so a member whose size the sector carries across a 256 KiB boundary
  now shows all four labels instead of three. And a geli provider is named as
  encrypted rather than reported as a disk with no ZFS on it.

  Checked on FreeBSD 15 in CI with the real tools: gpart add -l and
  glabel label on md devices, the GUID read back equal to what
  gpart list prints, and the volume out of the labelled member equal
  to the volume out of the plain one.

• The ends of the ashift range, against zdb. The fixtures were
  already built at ashift 9 and 13 as well as 12 (CI step "Every
  ashift"), but the cross-check against OpenZFS userland ran every
  geometry at 12. Two more ztest pools — raidz2 at ashift 13 and a
  mirror at 9 — now go through the same thirteen steps as the others,
  so the stride that decides whether a RAIDZ column read lands on the
  right sector is compared against zdb at both ends and not only at
  the middle. The real-world matrix rows say 9, 12 and 13 accordingly.

Full history: [CHANGELOG.md](https://github.com/click0/zvolrescue/blob/v0.8.5/CHANG...

zvolrescue v0.8.0 — A vdev that is gone, a dataset's own settings, and what a hash is for

zvolrescue v0.8.0 — read-only extraction of zvols from ZFS pools that no longer import: labels, uberblocks, mirror/RAIDZ/dRAID reconstruction, every checksum and compression, encrypted datasets with a key.

0.x release. Validated in CI against OpenZFS 2.2 userland pools (ztest + zdb) and on the environments listed in docs/REALWORLD-TESTS.md; treat the output as evidence to verify, not as a finished recovery.

Downloads

15 binaries: 5 programs, each built for the same three platforms. The name is always <program>-0.8.0-<platform>.

Program	What it is for
zvolrescue	the pool, and a volume out of it: what the members hold, and the image of a zvol
zvoltimeline	the pool history the transaction groups still carry: when a dataset was made, destroyed, and whether its space is still held
zvolcarve	volumes no uberblock points at any more, found by scanning raw vdev space
zvolfiles	files out of filesystem datasets: the tree with its metadata, or a raw object dump when the metadata is gone
zvolreport	the evidence logs of the others as one document, hashed, signed and checkable by a third party

Platform	<platform>	Notes
Linux x86_64	x86_64-linux-musl	static; runs on any distribution, and on a rescue medium
Linux aarch64	aarch64-linux-musl	static
FreeBSD 15.x amd64	amd64-freebsd	built on 15.x. FreeBSD changes its ABI between major branches, so on 14.x build from source: pkg install rust && cargo build --release

So the carver for Linux x86_64 is zvolcarve-0.8.0-x86_64-linux-musl, and the FreeBSD main binary is zvolrescue-0.8.0-amd64-freebsd.

SHA256SUMS covers all 15 of them:

sha256sum -c SHA256SUMS   # or, on FreeBSD: sha256 -c SHA256SUMS

No installation: chmod +x and run. Every program opens the members read-only and never writes to the pool.

What is in v0.8.0

A vdev that is gone, a dataset's own settings, and what a hash is for.
A pool a top-level vdev was removed from now reads, through the mapping
the removal left in the MOS, and a feature this build cannot account for
stops the read by name instead of letting it guess. list --properties
shows what a dataset was written with, user properties included. dump --hash md5,sha1 takes the legacy digests in the same pass as SHA-256
and zvolreport verify checks every digest a record carries back.
zvolcarve zeropoint and zvolcarve roots give two more ways into a
pool nothing points into. And an image dump had to fill with zeros is
exit 4, not a success.

Added

• dump --hash md5,sha1: the legacy digests, taken in the same pass
  (SPEC F-53). An extracted image usually leaves for somewhere else,
  and what is waiting there may be an acquisition log, a
  case-management system or a hashset that only speaks MD5 or SHA-1.
  Taking one afterwards costs another full read of a multi-terabyte
  image, so the moment to take it is while the bytes are going past —
  which is what the requirement said and what half of it had been doing
  for SHA-256 alone.

  Neither is offered as a check on whether the image is right. Both
  have practical collision attacks, which matters when someone may have
  chosen the bytes and does not matter for lining an image up against a
  record written before this tool touched it. That is what they are
  for, and the module says so where someone might otherwise reach for
  one as a verifier.

  What was taken goes into the evidence record, into the report, and
  into zvolreport verify, which now checks every digest a record
  carries rather than only its SHA-256 — one result per digest, with
  the file read once for all of them. A hash that is written down and
  never checked back is decoration.

  zvolcarve dump takes the same flag: it writes images too.

  Checked against the tools a forensic toolchain would actually use:
  sha256sum, sha1sum and md5sum agree with all three on a real
  32 MiB sparse image, the published RFC 1321 and FIPS 180 vectors are
  unit tests, and a resumed extraction feeds the prefix into each
  digest so that stopping and continuing comes to what one pass would
  have. One byte changed in a finished image, and all three say so.

• list --properties: what a dataset has set, user properties
  included (SPEC F-14). The properties ZAP was parsed as far as its
  object number and never read. It holds what an operator holding a
  dead pool most wants next after the dataset list: what compression
  and checksum this volume was written with, what recordsize it
  used, and whatever the shop's own bookkeeping put in a user property.

  Only what was set here is on disk. A property nobody set is
  inherited from an ancestor or is the pool's default, and neither is
  written down anywhere — so this reports what the dataset holds and
  says so, rather than computing an effective value out of a table of
  defaults that no disk would confirm. The ancestors are in the same
  listing, which is where an inherited value comes from.

  A number keeps its number unless the build can show its working.
  checksum and dedup carry the same zio_checksum enumeration a
  block pointer carries, which this tool already decodes on every block
  it reads and the cross-check compares against zdb. compression
  carries a zio_compress code in its low seven bits with a zstd
  level above them — and that split was measured, not remembered:
  across the five pools the cross-check builds and two more with vdevs
  removed, nineteen distinct values appear, and every one either fits
  in seven bits (on, lzjb, zle, lz4) or has exactly 16 — zstd — in
  them with 1..=19 or 100..=120 above. Nothing else produced a high
  part at all. Everything else — copies, recordsize, every on/off
  flag — prints the number the pool holds, because a name this build
  cannot evidence would be worse than the number.

  Checked against zdb on a real ztest pool: twenty properties over
  eight datasets, name for name and value for value, with an empty
  diff. In CI a fixture volume carries two of ZFS's own properties
  and a user property — which forces a fatzap, since a microzap entry
  is one 64-bit integer and cannot hold a string at all — and the
  listing is compared in both text and JSON. Asking is what separates
  "none set" from "not asked": without the flag no properties are read
  at all, and with it a dataset that set none shows an empty list.

• A pool a top-level vdev was removed from now reads (SPEC F-69).
  zpool remove does not free what was on a vdev: it copies those
  blocks onto the vdevs that remain and leaves in its place an
  indirect vdev holding nothing but a record of where each range of
  its old address space went. The block pointers are never rewritten —
  that would mean walking every pointer in the pool — so they go on
  naming a vdev that no longer exists, and every one of them was
  refused: DVA names unknown top-level vdev 3. It came up three times
  in one afternoon on ztest images that happened to have had a vdev
  removed during the run, which is what put it at the top of the list.

  That mapping is now read and the addresses translated. A range the
  removal copied in pieces is several entries and is joined back
  together; a piece that landed on a vdev itself removed later is
  followed through that one too, which real pools do — ztest produced
  one with three removed vdevs where one maps onto another. Where the
  destination is a mirror, each of its sides is still offered as a
  separate copy, so redundancy survives the translation.

  The mapping comes from the pool's own account of itself, not from
  the labels. No label describes a removed vdev: removing it is what
  took its members away. What does describe it is the configuration
  object in the MOS — and reading it there rather than anywhere else is
  also what keeps an older transaction group honest. Asked for a txg
  from before the removal, the configuration of that txg has no
  indirect vdev in it, and nothing is translated, because at that point
  nothing had been.

  scan asks rather than guesses. A removed vdev is still counted in
  vdev_children and still has no member to find, so from the labels
  alone it is a missing vdev — and a first version of this said exactly
  that, calling a pool with every live member present "NOT readable" in
  the summary line and readable: false in the JSON a script keys on.
  Now scan asks the pool's configuration object which undescribed
  vdevs were removed, only when there is one to ask about and
  device_removal is active (a healthy pool never opens its MOS for
  this), and reports them as removed with the pool readable. When the
  MOS cannot be reached it says so and leaves them missing, which they
  may well be. The same answer reaches --assume-member, which no
  longer counts a vdev that is gone on purpose against the leaf being
  bound.

  com.delphix:device_removal moves to the features this build honours
  and com.delphix:obsolete_counts deliberately does not. The counts
  are bookkeeping over the same mapping, and a pointer that still names
  the removed vdev is live whatever they say — but no pool measured
  against this build has had that feature active, and F-70's rule is
  that nothing is honoured here on reasoning alone.

  Checked against zdb on two real ztest pools with vdevs removed
  mid-run, one with three and one with four: the mapping objects match
  entry for entry and byte for byte, and every object comes out — 343
  of 343 and 279 of 279, with nothing refused. In CI a fixture whose
  volume is addressed entirely on a removed vdev extracts to the same
  sha256 as the same volume on a pool that never had one, and the same
  blocks are refused by name when the mapping is not...