v0.9.1

v0.9.1

For anyone running 0.9.0-beta.4 or an earlier pre-release. This release closes ways a file name or an
argument could reach the operator's terminal or files, and makes the commander fast inside a large
scan. The database stays at schema v6, so nothing is migrated. The number drops the -beta suffix;
the leading 0 still marks the project as a beta.

A file name can no longer drive the terminal

A file name holding an escape sequence could retitle the operator's terminal or clear the screen
when the commander listed it. Every screen now shows paths escaped, the status line is escaped again
where it is drawn, and each frame is checked before it reaches the terminal. In a CSV export, control
characters in a name are written as \n, \u{1b} and so on: printing the file runs nothing, and each
record stays one line. The reason a scan stopped, the --stats environment line and error messages
are escaped as well.

No writing through links or into a directory that is not dedcom's

• --export-csv onto a symlink replaced the symlink itself; as root, a typo could turn /dev/stdout
  into a regular file until reboot. It now refuses, and the export never writes through a link.
• dedcom.lock and consent.json were written through a symlink, a hardlink or a device node under
  their name. Only a regular file with exactly one name is accepted now, and the refusal names what
  was found.
• --state-dir took any directory and changed its permissions to 0700. An existing directory that
  holds neither dedcom's database nor its lock, but holds something, is now refused with an
  explanation, and so is an empty directory directly under /. A config.json put into a new
  directory by hand before the first start is refused too. --stats and --export-csv no longer
  create the state directory.

Two panics fixed

F3 in a window lower than five rows panicked and left the terminal in raw mode. dedcom --stats | head exited with 101 and a panic; it now exits quietly.

The commander inside a large scan

Changing directory inside a scanned tree took seconds of CPU per change: on a scan of 20,000 files,
about 3.8 s in beta.4, about 20 ms now, and F3, opening a group and marking no longer wait behind it.
At the top of a large scan the panel no longer reads the whole subtree on every refresh: the first
visit to a directory is one pass over its rows, and going back up or coming back is immediate while
the database does not change.

Marks show and do what the database holds

• "group files" and "duplicates of the cursor" show a mark as soon as it is saved; before, a new mark
  appeared only after the cursor left the group and came back.
• F9 → "Clear all marks" asks first and then clears every saved mark of the scan, keepers and marks
  set in an earlier session included. Before, it cleared one panel on screen while the database kept
  every mark, and F11 still built its plan from them.
• A mark on a path that is not valid UTF-8 is refused, and the status says why: a scan never records
  such a path, and the mark could land on another file whose path looks the same.

The manual

The backup and restore scripts in §12 work for databases from 0.9.0-beta.1 and beta.2, step 8 of the
quickstart can be followed with the keys it names, and the sample screens and the --purge-quarantine
output match the program.

What it does not establish

• Unicode formatting characters in names, such as bidirectional overrides and zero-width characters,
  are not escaped; a terminal with bidi support may reorder a row that shows one.
• A command-line argument that is not valid UTF-8 still stops dedcom with a panic at start-up.
• A plan saved as .sh holds the real bytes of the names, as it must to run, so cat of that file
  still passes an escape sequence in a name to the terminal. The Commands tab shows them escaped.
• The first visit to a very large directory still costs one pass over its rows, seconds at the top of
  a pool of millions of files.
• In "duplicates of the cursor" each step of the cursor over a file reads that file's group again:
  about 70 ms on a scan of a million files, on this project's test machine.
• A files panel shows only the marks set while it was open. Marks saved earlier show in "group files"
  and on the F11 confirmation.
• For a name that is not valid UTF-8, F3 and "duplicates of the cursor" can answer for another file
  whose name reads the same. Planned for 0.9.2.
• If the F2 check of saved scans answers while the F9 menu is open, its window replaces the menu, and
  an Enter meant for the menu opens that scan's results, or resumes its unfinished scan. Planned for
  0.9.2.
• The manual (§11.4) says a name that is not valid UTF-8 is exported with U+FFFD; a scan does not
  record such a name, so the export does not hold it. Corrected in 0.9.2.
• Hardlink and reflink refuse names longer than about 213 bytes, and a path longer than 4,096 bytes
  stops the scan. Both are planned for 0.9.2.

On upgrade

--storage-type accepts only hdd, ssd and nvme; any other value is now an error. A
--state-dir that dedcom does not recognise as its own is refused: point it to a new or empty
directory. beta.4 databases open as they are. From beta.1 to beta.3 the first start upgrades
dedcom.db to schema v6, as beta.4 did: back it up first with the procedure in the manual (§12,
"Backing up and restoring dedcom.db").

With this release, 0.9.0-beta.1 to 0.9.0-beta.4 are no longer supported.