v0.9.2

v0.9.2

For anyone running 0.9.1. This release checks every action before the first snapshot is taken, makes a
saved plan script as careful as applying with Y, stops a scan within about a second, keeps hashing
fast in a large scan, refuses command lines it would half-ignore, and makes mouse clicks and keys act
on what the screen shows. The database stays at schema v6, so nothing is migrated.

Applying actions

• Before the first snapshot, every action is checked: a read-only file system; chattr +i or +a on
  the file, its directory or the quarantine directory; no space or an exhausted quota; a file reached
  through a second mount (a bind mount) or lying outside its dataset's mount point; for a hardlink,
  the keeper as well. Such an action is refused with its reason and without reading any file; the
  others run. A snapshot is taken only of the datasets where there is work.
• If nothing in a batch can run, nothing changes and no snapshot is taken:
  nothing done — N actions cannot run: <reason>; no snapshot taken, marks kept; first: <path>, and
  the plan goes back to the window where it was confirmed.
• Reflink on a host without block cloning (OpenZFS older than 2.2.1, or zfs_bclone_enabled=0), or on
  a pool whose feature@block_cloning is disabled, is refused before the confirmation window, with
  the number of marks and what to do instead.
• A hardlink or reflink whose keeper is on another dataset is refused by the plan itself, before any
  snapshot or read: cannot hardlink or reflink across datasets (N marks) — …. Reflink across the
  datasets of one pool never worked; the manual no longer promises it.
• Hardlink and reflink of a file whose name is up to the 255-byte limit no longer fail with
  File name too long, and a CSV export can be written to a file with such a name.
• A failed replacement names the real cause: the prepared replacement vanished, no space, the quota,
  a read-only file system, an I/O error. If another file took the original's place, so that the
  original cannot go back, the message gives its exact path in the quarantine. A failed database
  write during a scan names the operating system's error, and a problem with the database file or
  the lock says what was found under that name.

A saved plan script

The script that F11 → S saves now takes the same steps as applying with Y:

• Before each action it compares the file with its keeper byte for byte (cmp) and leaves alone a
  file that has changed or become a symbolic link (dedcom: skip <file>: …).
• The replacement, a link or a clone, is made beside the file first. Only then does the file go to
  the quarantine, and it comes back if the replacement cannot be put in its place.
• A reflink copy gets the owner, mode, times, extended attributes and ACL of the original.
• An mv -n that moved nothing no longer counts as a deletion, and a file whose quarantine is on
  another mount (a bind mount, or a link to another file system) is refused before the move.
• One failed action no longer stops the rest. The script ends with
  dedcom: N of M actions not carried out — each is named above and exit code 1.
• The Commands tab shows one function call per action; the functions are defined at the top of the
  script.

Scanning

• A stop during hashing (Esc on the scan screen; Ctrl+C, SIGTERM or SIGHUP for --scan) takes
  effect within about a second, even in the middle of a large file. Before, the scan first finished
  reading every file of the current batch of up to 64: tens of minutes for a 500 GB disk image. The
  files being read at that moment get no hash and are read again from the start on resume.
• Hashing no longer slows down as a large scan goes on: 100,000 small files take 11 s instead of
  6 minutes on this project's test machine, and a scan of 2 million files on a home server hashes
  about 300 files per second instead of 14.
• --scan of a root that does not exist or cannot be read is refused with exit code 1 before the
  lock and the database are touched, instead of ending as an empty "successful" scan that trimmed the
  history. A resumed scan prints the settings it keeps and refuses a flag it would ignore;
  --no-resume starts over.
• A scan that finds no file under a root where earlier scans of the same roots found files, usually
  the empty mount point of a dataset that did not mount, no longer sends those scans to the trash and
  says History kept: ….
• A directory that cannot be opened (removed during the scan, a path longer than 4,095 bytes, no
  search permission, a disk error) is one walk error instead of the end of the scan: Omissions: … walk errors, ⚠ gaps in the interface, and a walk error: <path>: <reason> line in dedcom.log.
• --include-ext tar.gz finds photos.tar.gz, '*.JPG' means jpg on the command line and in
  presets.json, and a value with / is refused.

The command line and config.json

• An argument that is not valid UTF-8 no longer stops dedcom with a panic: the state directory and
  the export file accept such names, and a scan root gets a clear refusal.
• Two modes at once, a repeated --export-csv, or a flag the run does not read are refused with exit
  code 2 and an explanation, instead of half of the command being dropped. So are --verify,
  --merkle-dirs, --strict-verify and --no-resume beside --read-only. A window started with
  --read-only answers an action with (started with --read-only).
• Refusals show invisible direction-control characters escaped.
• A config.json that cannot be read (not JSON, not an object, not a regular file) is no longer
  rewritten and decides nothing. Each writing run says so in one line; automatic VACUUM and history
  trimming are skipped until the file is fixed, and the lock policy falls back to ask. A field of
  the wrong kind turns off only its own decision. config.json is now written by replacing the file,
  with mode 0600.

Clicks and keys act on what is drawn

• A click on a function-key number in the commander's bottom bar runs that key at any terminal
  width. Before, at widths such as 80, 100, 160 and 200 columns, eight of the twelve numbers ran the
  key to their left: "9 Menu" set a Delete mark, "11 Exec" quit.
• A click or a double-click in a list of groups or of a group's files, in the wizard and in the
  commander, scrolled or not, takes the row drawn under the pointer. Before, the wizard took the row
  below it, and a double-click saved the wrong keeper; in the commander such a click dropped the
  cursor. PgUp and PgDn move by the visible part of the list.
• In the wizard's group view, 1 opens Folders and 2 opens Files, as labelled.
• While the help, the start-up notice, the role prompt, a function-key window or a yes/no question
  is open, a click does nothing: close the window first. Before, a click under the help could set a
  Delete mark. On the Triage Board, a click no longer moves the focus while a receiver waits for its
  digit.
• The "Please wait" box is no longer drawn over a question or a window that is waiting for a key.
  Before, it could hide "Move to trash?" or "Purge from trash?", where Enter means yes.

A late answer does not take over an open window

• When F2, F3 or F11 answers after you opened the F9 menu, another window, the help or the Triage
  Board, its window no longer opens over them, and no scan starts from under them. The status line
  says what arrived and which key to press again ("Close this window, press F2 again: …").
• A scan result that finishes opening after you opened a menu, a window or the Triage Board, or moved
  to another screen, opens without switching the screen; the status line names the scan and says how
  to show it. The help does not count: the groups open under it. Before, the group view took over,
  and an Enter meant for the menu saved a keeper. A plan finished after you left the group view is
  dropped; r builds it again.

Names that are not valid UTF-8

• For a file whose name shows with a ?, F3 no longer shows another file's checksum and duplicates,
  "duplicates of the cursor" no longer shows another file's group, and a directory no longer shows
  another directory's total size. Both screens say instead that the scan has no such name, and why.
• The manual (§11.4) no longer says such a name is exported. It says where these files are counted
  (Omissions: in the --scan output, ⚠ gaps in the TUI) and how to ask about one file (F3 in
  the commander). The sample --scan output in §11.1 matches what the program prints.

The cursor in a large scan

In "duplicates of the cursor" a cursor step costs a fraction of a millisecond and no longer grows
with the scan; in 0.9.1 it was about 70 ms on a scan of a million files, on this project's test
machine. Building the F11 plan no longer walks the scan for every mark and group.

The manual

• A new §8.9 on backups and the stores of other programs (Time Machine, restic, borg, kopia, Arq,
  Duplicacy, Proxmox Backup Server, iPhone backups, virtual machine disks): do not scan a store, stop
  the program before applying, apply with Y rather than a saved script, verify with the program's
  own tools before cleaning anything. Wherever the manual cleans the quarantine, the command is
  dedcom --purge-quarantine --yes.
• §11 gives a ready cron line that runs at low priority and checks that the pool is mounted.
  Chapter 13 quotes every reason an action can be refused before the snapshots, and every line a
  saved script prints; a test checks the quotes against the program.

What it does not establish

• Unicode formatting characters in names, such as bidirectional overrides and zero-width characters,
  are not escaped on screen; a terminal with bidi support may reorder a row that shows one.
  Command-line refus...

v0.9.1

v0.9.1

For anyone running 0.9.0-beta.4 or an earlier pre-release. This release closes ways a file name or an
argument could reach the operator's terminal or files, and makes the commander fast inside a large
scan. The database stays at schema v6, so nothing is migrated. The number drops the -beta suffix;
the leading 0 still marks the project as a beta.

A file name can no longer drive the terminal

A file name holding an escape sequence could retitle the operator's terminal or clear the screen
when the commander listed it. Every screen now shows paths escaped, the status line is escaped again
where it is drawn, and each frame is checked before it reaches the terminal. In a CSV export, control
characters in a name are written as \n, \u{1b} and so on: printing the file runs nothing, and each
record stays one line. The reason a scan stopped, the --stats environment line and error messages
are escaped as well.

No writing through links or into a directory that is not dedcom's

• --export-csv onto a symlink replaced the symlink itself; as root, a typo could turn /dev/stdout
  into a regular file until reboot. It now refuses, and the export never writes through a link.
• dedcom.lock and consent.json were written through a symlink, a hardlink or a device node under
  their name. Only a regular file with exactly one name is accepted now, and the refusal names what
  was found.
• --state-dir took any directory and changed its permissions to 0700. An existing directory that
  holds neither dedcom's database nor its lock, but holds something, is now refused with an
  explanation, and so is an empty directory directly under /. A config.json put into a new
  directory by hand before the first start is refused too. --stats and --export-csv no longer
  create the state directory.

Two panics fixed

F3 in a window lower than five rows panicked and left the terminal in raw mode. dedcom --stats | head exited with 101 and a panic; it now exits quietly.

The commander inside a large scan

Changing directory inside a scanned tree took seconds of CPU per change: on a scan of 20,000 files,
about 3.8 s in beta.4, about 20 ms now, and F3, opening a group and marking no longer wait behind it.
At the top of a large scan the panel no longer reads the whole subtree on every refresh: the first
visit to a directory is one pass over its rows, and going back up or coming back is immediate while
the database does not change.

Marks show and do what the database holds

• "group files" and "duplicates of the cursor" show a mark as soon as it is saved; before, a new mark
  appeared only after the cursor left the group and came back.
• F9 → "Clear all marks" asks first and then clears every saved mark of the scan, keepers and marks
  set in an earlier session included. Before, it cleared one panel on screen while the database kept
  every mark, and F11 still built its plan from them.
• A mark on a path that is not valid UTF-8 is refused, and the status says why: a scan never records
  such a path, and the mark could land on another file whose path looks the same.

The manual

The backup and restore scripts in §12 work for databases from 0.9.0-beta.1 and beta.2, step 8 of the
quickstart can be followed with the keys it names, and the sample screens and the --purge-quarantine
output match the program.

What it does not establish

• Unicode formatting characters in names, such as bidirectional overrides and zero-width characters,
  are not escaped; a terminal with bidi support may reorder a row that shows one.
• A command-line argument that is not valid UTF-8 still stops dedcom with a panic at start-up.
• A plan saved as .sh holds the real bytes of the names, as it must to run, so cat of that file
  still passes an escape sequence in a name to the terminal. The Commands tab shows them escaped.
• The first visit to a very large directory still costs one pass over its rows, seconds at the top of
  a pool of millions of files.
• In "duplicates of the cursor" each step of the cursor over a file reads that file's group again:
  about 70 ms on a scan of a million files, on this project's test machine.
• A files panel shows only the marks set while it was open. Marks saved earlier show in "group files"
  and on the F11 confirmation.
• For a name that is not valid UTF-8, F3 and "duplicates of the cursor" can answer for another file
  whose name reads the same. Planned for 0.9.2.
• If the F2 check of saved scans answers while the F9 menu is open, its window replaces the menu, and
  an Enter meant for the menu opens that scan's results, or resumes its unfinished scan. Planned for
  0.9.2.
• The manual (§11.4) says a name that is not valid UTF-8 is exported with U+FFFD; a scan does not
  record such a name, so the export does not hold it. Corrected in 0.9.2.
• Hardlink and reflink refuse names longer than about 213 bytes, and a path longer than 4,096 bytes
  stops the scan. Both are planned for 0.9.2.

On upgrade

--storage-type accepts only hdd, ssd and nvme; any other value is now an error. A
--state-dir that dedcom does not recognise as its own is refused: point it to a new or empty
directory. beta.4 databases open as they are. From beta.1 to beta.3 the first start upgrades
dedcom.db to schema v6, as beta.4 did: back it up first with the procedure in the manual (§12,
"Backing up and restoring dedcom.db").

With this release, 0.9.0-beta.1 to 0.9.0-beta.4 are no longer supported.

v0.9.0-beta.4

v0.9.0-beta.4

For anyone running 0.9.0-beta.3. Most of this release is about the checkpoint database,
dedcom.db: how its move journal stores pathnames, and what the shape guard accepts before it
writes anything. One fix changes what you see on disk and in the duplicate list.

Order and names on disk no longer depend on the machine

Two habits made one tree come out differently on two machines. Directory order was taken from
readdir, and a file name was rebuilt through a lossy string that maps distinct bytes onto one.
Siblings are now ordered by their raw bytes, and a name keeps the bytes it had. A failed read no
longer counts as an answer to whether two files are duplicates, and a merge that leaves the source
behind is not reported as clean.

The checkpoint moves to schema v6

The first start of this build upgrades dedcom.db to schema v6 in one transaction. The
move_event journal used to pass pathnames through a lossy UTF-8 conversion, so two names that
differ only in bytes outside UTF-8 collapsed into one record. v6 stores both pathnames as raw bytes
and marks each row with path_fidelity: 1 for rows this build wrote, 0 for rows carried over from
the v5 journal.

The table is rebuilt only after its stored definition and everything attached to it (indexes,
triggers, views, foreign keys) prove to be what this product wrote. Anything else stops the upgrade
with a message, and the data and the schema version stay as they were.

The shape guard refuses more before it writes

Before the database is switched to WAL, the guard now refuses:

• a product name held in another letter case. SQLite resolves names without regard to ASCII case,
  the guard compared them byte for byte, and a foreign SCAN_ROOT could be adopted;
• two objects under one product name, such as a table and a trigger;
• a hidden or generated column under a product column name, and a column of a later schema in a
  checkpoint that declares an older one;
• a virtual or shadow table (FTS, R*Tree and the like) under a product table name. It could carry
  the right column names and be taken for the product's own.

A refused file keeps its bytes. Objects under names the product does not use, virtual tables
included, are left alone.

Also in this release

The manual matches this release: the F9 menu, the wording of the size figure on the confirmation
and summary screens, and the session-retention default. A test now holds the manual to what the
code does. Two paths gained tests: the order of a walk over several roots that cannot be keyed, and
a refused stat while listing files of the same size.

What it does not establish

• Journal rows carried over from v5 keep the text v5 had. Bytes lost then are not recovered, and
  those rows are not claimed to be exact.
• The journal is still best-effort: a move whose record could not be written is not reported.
• The declared types and defaults of required columns are not compared, and a WITHOUT ROWID or
  STRICT table under a product name passes as an ordinary table.

On upgrade

Back up dedcom.db first with the procedure in the manual (§12, "Backing up and restoring
dedcom.db"). A plain cp misses the -wal file. beta.2 and beta.3 refuse to open a v6 database
before writing to it, so going back means restoring that backup.

v0.9.0-beta.3

G0–G3 passed. G4 product behaviour was not assessed because the integration harness was
incompatible with the execution environment. No candidate mismatch was observed.

Unstable pre-release. The ZFS end-to-end path is unverified for this build.

Known: two pipeline::walk tests assert a directory traversal order that is not guaranteed;
they fail on the amd64 CI runner and pass on arm64. Test-side defect, tracked separately.

---

Full Changelog: v0.9.0-beta.2...v0.9.0-beta.3

v0.9.0-beta.2

What's Changed

• fix(commander): guard row commands by panel view [U-1] by @dequzzie in #2
• fix(deps): bump crossbeam-epoch to close RUSTSEC-2026-0204 by @dequzzie in #4
• fix(store): version the schema and skip unknown statuses [S-3] by @dequzzie in #3
• fix(store): prepare finished scans on the writer path by @dequzzie in #5
• chore(release): 0.9.0-beta.2 by @dequzzie in #6
• fix(store): make read-only a real read-only connection [S-1/A-1] by @dequzzie in #7
• fix(lock): fail closed when the lock cannot be evaluated [S-2] by @dequzzie in #8
• fix(actions): drop plan targets already identical to their keeper [D-4] by @dequzzie in #9
• feat(signals): stop on SIGINT/SIGTERM/SIGHUP at an action boundary [A-3] by @dequzzie in #10
• fix(workers): report a panicking worker instead of leaving the screen… by @dequzzie in #11
• fix(actions): report a cancelled batch instead of a finished one [D-6] by @dequzzie in #12
• fix(reflink): keep the target's owner/mode/xattr on the clone [D-1] by @dequzzie in #13
• feat(commander): reach Execute without F11 [U-2] by @dequzzie in #14
• fix(review): scroll the action list so the whole plan is readable [U-3] by @dequzzie in #15
• feat(commander): say which actions the batch runs, and on what [U-4a] by @dequzzie in #16
• feat(commander): scroll the F11 command list to its end [U-4b] by @dequzzie in #17
• fix(commander): stop Enter from executing the batch [U-4c] by @dequzzie in #18
• test(bench): assert the bench record instead of the tracing output [C… by @dequzzie in https://github.com//pull/19

New Contributors

• @dequzzie made their first contribution in #2

Full Changelog: v0.9.0-beta.1...v0.9.0-beta.2

v0.9.0-beta.1

Full Changelog: https://github.com/dedupcommando/DedupCommando/commits/v0.9.0-beta.1

Full Changelog: https://github.com/dedupcommando/DedupCommando/commits/v0.9.0-beta.1