Repository navigation
Compare two scans with the fs_diff DuckDB macros, much faster scans of very deep trees, and directories deleted while they're read counted as vanished instead of as errors. New tests check the output against GNU find, scans of a tree that changes throughout, random names through every text encoder, and trees 2,000 levels deep or with a million entries in one directory.
From 1.0 on, the command line, the output tables and their columns, the manifest, and the DuckDB macros follow semantic versioning: 1.x releases can add options, columns (at the end of a table, with a new schema_version), and macros, but won't remove or rename them.
Added
fs_diffandfs_diff_summaryDuckDB macros compare two Parquet scans of the same roots: one row per added, removed, or changed entry, with the columns that changed and the rows before and after, or the counts and net size change by kind. Entries are matched by their exact path bytes;atimeanddevare ignored unlessignore := []is given. They work on version 2 scans too.
Changed
converttakes its output format with-f/--format, likescan.--to, its name in 0.6.0, still works.
Fixed
- Directories whose paths are longer than
PATH_MAX(4,096 bytes on Linux, 1,024 on macOS) are opened in a few calls instead of one call per path component. Opening every directory of a 2,000-level tree took millions of calls: scanning one on macOS went from 7.5 minutes to 6 seconds. - A directory deleted while it was being read counts as vanished, like other deletions during a scan, instead of as a
readdirerror (Linux reports it asENOENT). Busy file systems no longer end a scan with exit code 1 for routine churn.
Full changelog: v0.6.0...v1.0.0